如何在不读取整行或整个文件的情况下读取令牌

从文件中逐个读取令牌；您可以使用

re

#!/usr/bin/env python3import reimport sysfrom mmap import ACCESS_READ, mmapdef generate_tokens(filename, pattern):    with open(filename) as f, mmap(f.fileno(), 0, access=ACCESS_READ) as mm:         yield from re.finditer(pattern, mm)# sum all integers in a file specified at the command-lineprint(sum(int(m.group()) for m in generate_tokens(sys.argv[1], br'd+')))

即使文件不适合内存，它也可以工作。