Script Development - Major Scripts

structure of how major scripts working.

Introduction

This page introduces some major scripts I developed for this project. I have learned a lot during this process. Due to the number of scripts and also the security concerns, more details will not discussed.

List of Major Scripts

Script Name Purpose Size
7zcomp.py Compressing batches of files using 7zip. 11 KB
analyze_single.py Analyze a single file with the analysis pipeline. (multithreading) 61 KB
analyze_single_serialize.py Analyze a single file using a single core with the analysis pipeline. 68 KB
bulkexport.py Extract links from a batch of accessions. (multithreading) 15 KB
bulkstore.py Initiate storage pipeline, invoking necessary scripts with proper parameters. 10 KB
correct_errors.py Trying to fix error files. 13 KB
helper.py Contains all helper functions. 29 KB
merge_result.py Combining results of different files in each accession. 9 KB
movecomplete.py Move and validate downloaded files. 11 KB
proc_error.py The wrapper of error processing, invoking different required scripts. 16 KB
recdown.py Retrieve files from a remote proxy node. 14 KB
startdown.py Download data from ENA's public server. 8 KB
start_operation.py Entrance of all operations. This wrapper invokes required scripts using proper parameters. 20 KB
startanalyze.py The master scheduler of the analysis pipeline. 16 KB
switch_down.py Switch the download list for proxy nodes, and clean any residual left. 29 KB
get_information.py Get information from the database, calculate values, and present in a human-readable format. 22 KB

Navigate through the Script Development Section

Navigate through the Genetic Project