cromwell-web server 服务器端

cromwell 除了之前提到的执行单个任务流程的run 模式，还有强大的server模式，即启动一个web服务，然后通过接收REST 请求实现任务调度。

java -Dconfig.file=cromwell.config -jar cromwell-69.jar server

cromwell默认开启8000 端口，可以在cromwell.config中修改网络配置。

访问 http://127.0.0.1:8000/swagger/index.html?url=/swagger/cromwell.yaml#/

页面提供了很多对任务（流程）的提交、查询、统计等API，可以在网页中直接点击提交，也可以使用curl/wget 等命令行进行处理。

值得兴奋的是，很多组织对这些API进行了进一步的封装，比较流行的是cromshell、cromwell-tools、oliver

cromshell

cromshell 由broad开发，实质是基于curl 的shell脚本做的进一步封装。

https://github.com/broadinstitute/cromshell

子命令: Start/Stop workflows

submit [-w] [options_json] [included_wdl_zip_file]
- Submit a new workflow.
- Will automatically validate the WDL and JSON file if womtool is in your path.
- -w Wait for workflow to transition from ‘Submitted’ to some other status before ${scriptNAME} exits
- included_wdl_zip_file Zip file containing any WDL files included in the input WDL
  需要注意的是，如果included_wdl_zip_file 存在那么options_json也必须填写，可以做一个空文件。
abort [workflow-id] [[workflow-id]...]
- Abort a running workflow.

Workflow information:

alias
- Label the given workflow ID with the given alias_name. Aliases can be used in place of workflow IDs to reference jobs.

Query workflow status:

status [workflow-id] [[workflow-id]...]
- Check the status of a workflow.
metadata [workflow-id] [[workflow-id]...]
- Get the full metadata of a workflow.
slim-metadata [workflow-id] [[workflow-id]...]
- Get a subset of the metadata from a workflow.
execution-status-count, counts [-p] [-x] [workflow-id] [[workflow-id]...]
- Get the summarized status of all jobs in the workflow.
- -p prints a pretty summary of the execution status instead of JSON
- -x expands sub-workflows for more detailed summarization
timing [workflow-id] [[workflow-id]...]
- Open the timing diagram in a browser.

Logs

logs [workflow-id] [[workflow-id]...]
- List the log files produced by a workflow.
fetch-logs [workflow-id] [[workflow-id]...]
- Download all logs produced by a workflow.

Job Outputs

list-outputs [workflow-id] [[workflow-id]...]
- List all output files produced by a workflow.
fetch-all [workflow-id] [[workflow-id]...]
- Download all output files produced by a workflow.

Get email notification on job completion

notify [workflow-id] [daemon-server] email [cromwell-server]
- daemon-server server to run the notification daemon on

Display a list jobs submitted through cromshell

list [-c] [-u]
- -c Color the output by completion status
- -u Check completion status of all unfinished jobs

Clean up local cached list

cleanup [-s STATUS]
- Remove completed jobs from local list.
  Will remove all jobs from the local list that are in a completed state,
  where a completed state is one of: Succeeded, Failed, Aborted
- -s STATUS If provided, will only remove jobs with the given STATUS from the local list.

测试过上面的命令，基本可以满足我们的日常需求。

cromwell-tools

cromshell 也是由broad开发，实质是基于python 对API的进一步封装。感兴趣可以参考 https://github.com/broadinstitute/cromwell-tools

$> cromwell-tools -h
usage: cromwell-tools [-h]
                  {submit,wait,status,abort,release_hold,query,health}
                  ...

positional arguments:
  {submit,wait,status,abort,release_hold,query,health}
                        sub-command help
    submit              submit help
    wait                wait help
    status              status help
    abort               abort help
    release_hold        release_hold help
    query               query help
    health              health help

oliver

oliver 是 St. Jude Children’s Research Hospital（https://github.com/stjudecloud）基于python开发的cromwell工具.

相比于以上两个工具，oliver对参数的处理更加方便，很多参数可以在json中添加，也可以在命令行中添加。

更多信息可以访问
https://github.com/stjudecloud/oliver

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-J4wJqBRa-1633584282413)(https://stjudecloud.github.io/oliver/images/oliver-example.gif)]

web界面

以上3个工具均是在命令行水平进行任务处理，但cromwell server的本质应该是提供更加友好的web界面，方便没有linux基础的人员进行操作，目前这方面的工具还不完善，仅有几个工具，没有详细调研供参考, 后面可能会另开一个页面进行专门的介绍。

https://github.com/DataBiosphere/job-manager  
https://github.com/BiRG/cromwell-frontend
https://github.com/yunhailuo/cromwell-authNboard # 感觉最好的

cromwell-web server 服务器端

Python相关栏目本月热门文章