在使用pandas过程由于文本中存在形如, 、| 等常规字符 所以需要自定义分隔符 特别是自定义由多个字符组成的分隔符。那么此时在使用 pandas.read_csv()的时候要如何设置
解决比如当生成文件的时候使用#|#作为分隔符 直接使用df pd.read_csv(raw_file, sep #|# , quoting 3)会报错
df pd.read_csv(raw_file, sep #|# , quoting 3) File /data/miniconda3/envs/python36/lib/python3.6/site-packages/pandas/io/parsers.py , line 688, in read_csv return _read(filepath_or_buffer, kwds) File /data/miniconda3/envs/python36/lib/python3.6/site-packages/pandas/io/parsers.py , line 460, in _read data parser.read(nrows) File /data/miniconda3/envs/python36/lib/python3.6/site-packages/pandas/io/parsers.py , line 1198, in read ret self._engine.read(nrows) File /data/miniconda3/envs/python36/lib/python3.6/site-packages/pandas/io/parsers.py , line 2585, in read alldata self._rows_to_cols(content) File /data/miniconda3/envs/python36/lib/python3.6/site-packages/pandas/io/parsers.py , line 3237, in _rows_to_cols self._alert_malformed(msg, row_num 1) File /data/miniconda3/envs/python36/lib/python3.6/site-packages/pandas/io/parsers.py , line 2998, in _alert_malformed raise ParserError(msg)
需要将其改为
df pd.read_csv(raw_file, sep #|# , quoting 3)
官方文档是这么说明的
In addition, separators longer than 1 character and different from s will be interpreted as regular expressions and will also force the use of the Python parsing engine. Note that regex delimiters are prone to ignoring quoted data. Regex example: rt .
