summaryrefslogtreecommitdiff
path: root/Code/download_and_map.py
diff options
context:
space:
mode:
authorHui Lan <lanhui@zjnu.edu.cn>2020-01-21 11:52:09 +0800
committerHui Lan <lanhui@zjnu.edu.cn>2020-01-21 11:52:09 +0800
commit6701b310b0f2694b681517381ad889ec67012fe0 (patch)
treefea6d914a490fafd5d67aff5d5498813d6849cfb /Code/download_and_map.py
parent5a70b3b498e64dc903b017d45be09a808cfb2b89 (diff)
merge_edges.py: write all edge information to an SQLite database file called edges.sqlite
When I saved a static html page for each edge (e.g., http://118.25.96.118/static/edges/AT1G20910_AT1G30100_0.html), it took 5GB disk space for saving 1 million html pages. Not very disk space efficient. An alternative is to save all edge information in a database table (i.e., edge), and query this database table for a particular edge. The database file edges.sqlite takes less than 200MB for 1 million edges, requiring 10 times smaller space than the static approach. The reason is that we do not have a lot of HTML tags in the database. Quite happy about that, though it seems that filling a database is a bit slower (2 hours??? for 1 million rows). Also updated two files that were affected: update_network.py and update_network_by_force.py. Now instead of copying 1 million static html page to the Webapp, I just need to copy edges.sqlite to static/edges/. Faster. In the Webapp, I updated start_webapp.py and added a file templates/edge.html for handling dynamic page generation. -Hui
Diffstat (limited to 'Code/download_and_map.py')
0 files changed, 0 insertions, 0 deletions