源码下载:
GitHub - Strawberry-Eat-Mango/PCT_Pytorch: Pytorch implementation of PCT: Point Cloud Transformer问题:
很难找到在Win10下的复杂混合编译案例。
解决方案:
依据执行python setup.py build命令后的错误提示信息(缺头文件、库文件、无法找到文件、无法找到cl等等),做下述修改。
1、编译环境
Win10,VS2017
参考
https://zhuanlan.zhihu.com/p/371279126
设置vc环境vcx64.bat ,set DISTUTILS_USE_SDK=1 。
安装ninja,引导cpp_extension.py进入unix编译环境,分别调用nvcc编译器和cl编译器。
否则将进入gcc编译器,感觉Win10环境下问题更多。
2、尝试修改cpp_extension.py
原始编译过程中出现错误: cl /showIncludes -mdll -O -Wall -DMS_WIN64 '-IE:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libpointnet2_ops_ext-srcinclude' '-ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchinclude' '-ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludetorchcsrcapiinclude' '-ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludeTH' '-ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludeTHC' '-ID:ProgramTensorNVIDIACUDAv10.0include' '-ID:ProgramTensorAnaconda3envspytorchinclude' '-ID:ProgramTensorAnaconda3envspytorchinclude' -c -c E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libpointnet2_ops_ext-srcsrcgroup_points.cpp /FoE:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcgroup_points.o -Zi -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14 显然,其中包含头文件-I选项出现了错误(-I不能出现在表示路径的引号之内) 解决方案: 进入cpp_extension.cpp,定位_write_ninja_file_and_compile_objects函数(约1225行),其中cflags为cl编译选项,post_cflags为cl编译后置选项,cuda_cflags,cuda_post_cflags为nvcc编译的对应选项。
在调用_write_ninja_file函数之前,编写命令行字符串处理函数:
def formatcflags(cflags):
temp = []
for f in cflags:
if f.count('-ccbin '):
temp.append('-ccbin ')
f = f.replace('-I-ccbin ', '')
f = f[1: -1]
f = '"' + f + '"'
elif f.count('-I'):
temp.append('-I')
f = f.replace('-I', '')
f = f[1: -1]
f = '"' + f + '"'
temp.append(f)
temp = [f.replace('\', '/') for f in temp]
return temp
def formatcudapostflags(pflags):
#去除-fPIC选项前后多余的单引号和双引号
pflags = [f[6:-6] if f.count('-fPIC') else f for f in pflags]
return pflags
调用以上函数,分别处理cflags、cuda_cflags和cuda_post_flags:
cflags = formatcflags(cflags) cuda_cflags = formatcflags(cuda_cflags) cuda_post_cflags = formatcudapostflags(cuda_post_cflags)
3、尝试修改setup.py
常常会出现缺少头文件、库文件的情况,因此需要对setup.py做附加信息处理,定义:
cmd1 = r'-ccbin D:Program Files (x86)Microsoft Visual Studio2017ProfessionalVCToolsMSVC14.16.27023binHostx64x64' vcdir1 = r'C:/Program Files (x86)/Windows Kits/10/Include/10.0.17763.0/shared' vcdir2 = r'C:/Program Files (x86)/Windows Kits/10/Include/10.0.17763.0/ucrt' vcdir3 = r'D:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/include'
然后把这些信息粗暴的加入到include中:
include_dirs=[osp.join(this_dir, _ext_src_root, "include"), cmd1, vcdir1, vcdir2, vcdir3]
加入的上述信息,均会在前述字符串处理函数中加以处理,解析出 -I“目录” -ccbin“目录” 等Windows认可的编译命令形式。
4、链接
上述修改后,可以使用python setup.py build一次性完成9个文件(.cpp .cu)的编译,生成9个.o文件和一个.def文件,但是会出现g++链接错误。很显然上述修改只是粗暴地改变了编译选项(但是实现了多个文件一次性编译,不必分9次编译),对链接没有影响,所以系统仍然沿循g++ 、gcc链接命令。
既然Windows,还是要引入到link命令上去。
限于时间作者没有尝试如何修改引导编译、链接分别进入Windows正确的轨道。只好采用单独执行一次link命令的方法进行链接:
link -dll E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcball_query.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcbindings.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcgroup_points.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcinterpolate.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcsampling.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcball_query_gpu.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcgroup_points_gpu.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcinterpolate_gpu.o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcsampling_gpu.o /libpath:"D:Program Files (x86)Microsoft Visual Studio2017ProfessionalVCToolsMSVC14.16.27023libonecorex64" /libpath:"D:ProgramTensorAnaconda3libs" /libpath:"C:Program Files (x86)Windows Kits10Lib10.0.17763.0umx64" /libpath:"C:Program Files (x86)Windows Kits10Lib10.0.17763.0ucrtx64" -LIBPATH:D:ProgramTensorAnaconda3envspytorchlibsite-packagestorchlib -LIBPATH:D:ProgramTensorNVIDIACUDAv10.0lib/x64 -LIBPATH:D:ProgramTensorAnaconda3envspytorchlibs -LIBPATH:D:ProgramTensorAnaconda3envspytorchPCbuildamd64 c10.lib torch.lib torch_cpu.lib torch_python.lib cudart.lib c10_cuda.lib torch_cuda.lib python37.lib -OUT:E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildlib.win-amd64-3.7pointnet2_ops_ext.cp37-win_amd64.pyd
链接的结果:
在build目录下生成_ext.cp37-win_amd64.pyd(动态库) .exp .lib文件
5、执行
将build下的pointnet2_ops文件夹整体拷贝至与main.py同级,python编译时即可找到
pointnet2_ops下的pointnet2_utils,并引用下述动态库包
import pointnet2_ops._ext as _ext
6、附:单条编译命令
如前所述,也可以采用逐个文件编译、整体链接的方式,这样的好处是完全抛弃setup.py,不必对源程序做任何修改,缺点是必须逐条编译,当然高手也可以写一个bat一次性执行。
[1/9]sampling_gpu.cu
[2/9]group_points_gpu.cu
[3/9]ball_query_gpu.cu
[4/9]interpolate_gpu.cu
[5/9]bindings.cpp
[5/9]group_points.cpp
[5/9]ball_query.cpp
[5/9]sampling.cpp
[5/9]interpolate.cpp
cu文件使用nvcc编译:
D:ProgramTensorNVIDIACUDAv10.0binnvcc -ccbin "D:Program Files (x86)Microsoft Visual Studio2017ProfessionalVCToolsMSVC14.16.27023binHostx64x64" -IE:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libpointnet2_ops_ext-srcinclude -ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchinclude -ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludetorchcsrcapiinclude -ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludeTH -ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludeTHC -ID:ProgramTensorNVIDIACUDAv10.0include -ID:ProgramTensorAnaconda3envspytorchinclude -ID:ProgramTensorAnaconda3envspytorchinclude -c -c E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libpointnet2_ops_ext-srcsrcsampling_gpu.cu -o E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcsampling_gpu.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options " " -fPIC " " -O3 -Xfatbin -compress-all -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=compute_37 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_62,code=sm_62 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_37,code=sm_37 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_50,code=sm_50 -std=c++14
cpp文件使用cl编译:
cl /showIncludes -Wall -DMS_WIN64 -I "C:Program Files (x86)Windows Kits10Include10.0.17763.0shared" -I"C:Program Files (x86)Windows Kits10Include10.0.17763.0ucrt" -I"D:Program Files (x86)Microsoft Visual Studio2017ProfessionalVCToolsMSVC14.16.27023include" -IE:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libpointnet2_ops_ext-srcinclude -ID:ProgramTensorAnaconda3envspytorchlibsite-packagestorchinclude -I"D:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludetorchcsrcapiinclude" -I"D:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludeTH" -I"D:ProgramTensorAnaconda3envspytorchlibsite-packagestorchincludeTHC" -I"D:ProgramTensorNVIDIACUDAv10.0include" -I"D:ProgramTensorAnaconda3envspytorchinclude" -I"D:ProgramTensorAnaconda3envspytorchinclude" -c -c E:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libpointnet2_ops_ext-srcsrcsampling.cpp /FoE:ZTensorprogramsPCT_Pytorch-mainpointnet2_ops_libbuildtemp.win-amd64-3.7Releasepointnet2_ops_ext-srcsrcsampling.o -Zi -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0
当然,也可以安装ubantu或linux系统,应该能够避免很多混合编译链接问题。
顺便地,源程序中下载H5格式的Modelnet40数据集,使用的wget、unzip、mv、rm等命令,常规下Windows是没有这些命令的:
#os.system('wget %s; unzip %s' % (www, zipfile))
可以下载安装wget等命令模拟Linux环境,当然也可以直接修改成Windows下已有的命令:
os.system('curl -k -O %s' % www) #wget
os.system('tar -xf %s' % zipfile) #unzip
os.system('move %s %s' % (zipfile[:-4], DATA_DIR)) #mv
os.system('del %s' % zipfile) #rm
申明:python菜鸟,不喜勿喷!!!



