这是一个使用pyarrow序列化熊猫数据帧以存储在Redis中的完整示例
apt-get install python3 python3-pip redis-serverpip3 install pandas pyarrow redis
然后在python中
import pandas as pdimport pyarrow as paimport redisdf=pd.Dataframe({'A':[1,2,3]})r = redis.Redis(host='localhost', port=6379, db=0)context = pa.default_serialization_context()r.set("key", context.serialize(df).to_buffer().to_pybytes())context.deserialize(r.get("key")) A0 11 22 3我刚刚向熊猫提交了PR 28494，以便在文档中包含这个pyarrow示例。
参考文档：
- https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Dataframe.to_msgpack.html
- https://arrow.apache.org/docs/python/ipc.html#arbitrary-object-serialization
- https://arrow.apache.org/docs/python/memory.html#pyarrow-buffer
- https://stackoverflow.com/a/37957490/4126114
