栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Python

python实现文件断点下载

Python 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

python实现文件断点下载

1.需求: 实现文件的断点下载 2.实现:
#!/usr/bin/python
# encoding=utf-8
import requests, sys, os, re, time


class download:
    def __init__(self, config={}):
        self.config = {
            'block': int(config['block'] if 'block' in config else 1024),
        }
        self.total = 0
        self.size = 0
        self.filename = ''

    def touch_file(self, filename):
        with open(filename, 'w') as f:
            pass

    def remove_nonchars(self, name):
        (name, _) = re.subn(r'[\/:*?"<>|]', '', name)
        return name

    def support_continue(self, url):
        headers = {'Range': 'bytes=0-4'}
        try:
            res = requests.head(url, headers=headers)
            crange = res.headers['content-range']
            self.total = int(re.match(r'^bytes 0-4/(d+)$', crange).group(1))
            return True
        except:
            pass
        try:
            self.total = int(res.headers['content-length'])
        except:
            self.total = 0
        return False

    def download(self, url, filename, headers={}):
        finished = False
        block = self.config['block']
        local_filename = self.remove_nonchars(filename)  # 移除文件名称中的特殊字符
        tmp_filename = local_filename + '.tmp'
        size = self.size
        if self.support_continue(url):  # 支持断点续传
            try:
                with open(tmp_filename, 'rb') as f:
                    self.size = int(f.read())
                    size = self.size + 1
            except:
                self.touch_file(tmp_filename)
            finally:
                headers['Range'] = "bytes=%d-" % self.size
                sys.stdout.write('nlast download size: %s!nthe start index for this time is: %sn' % (self.size, size))
                sys.stdout.flush()
        else:
            self.touch_file(tmp_filename)
            self.touch_file(local_filename)

        try:
            res = requests.get(url, stream=True, verify=False, headers=headers)
        except Exception as e:
            sys.stdout.write('ndownload failed: %s!n' % (str(e)))
            sys.stdout.flush()
            os.remove(tmp_filename)
            os.remove(local_filename)
            return

        start_t = time.time()
        with open(local_filename, 'ab+') as f:
            f.seek(self.size)
            f.truncate()
            try:
                for chunk in res.iter_content(chunk_size=block):
                    if chunk:
                        f.write(chunk)
                        size += len(chunk)
                        f.flush()
                    sys.stdout.write('b' * 64 + 'Now: %d, Total: %s' % (size, self.total))
                    sys.stdout.flush()
                finished = True
                os.remove(tmp_filename)
                spend = int(time.time() - start_t)
                speed = int((size - self.size) / 1024 / spend)
                sys.stdout.write('nDownload Finished!nTotal Time: %ss, Download Speed: %sk/sn' % (spend, speed))
                sys.stdout.flush()
            except Exception as e:
                sys.stdout.write("nDownload pause, reason: %sn", str(e))
                sys.stdout.flush()
            finally:
                if not finished:
                    with open(tmp_filename, 'wb') as tmp:
                        tmp.write(bytes(str(size), encoding="utf-8"))


if __name__ == '__main__':
    url = "https://yos.yingzi.com/prod-bucket-iot/prod-iot-storage/979743213606400000/BH220_RK_123025080.diff"
    filename = url.split('/')[-1]
    download().download(url, filename)



转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/934273.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号