Age | Commit message (Collapse) | Author | |
---|---|---|---|
2020-10-10 | buildRmatrix.py: use string instead of float to store TPM values. This ↵ | Hui Lan | |
script could run out of memory space (7GB) when there are more than 1000 RNA-seq samples. | |||
2020-08-27 | buildRmatrix.py: get max value eariler. | Hui Lan | |
2020-08-27 | buildRmatrix.py: remove a redundant line. | Hui Lan | |
2020-08-16 | html_network.py: make URL for edge information such as ↵ | Hui Lan | |
AT3G09600_AT1G49720_0.html correct (revision 2). | |||
2020-08-15 | html_network.py: make URL for edge information such as ↵ | Hui Lan | |
AT3G09600_AT1G49720_0.html correct (revision). | |||
2020-08-14 | start_webapp.py and util_networkx.py: remove them (they are in ↵ | Hui Lan | |
/var/www/brain/brain). | |||
2020-08-14 | html_network.py: make URL for edge information such as ↵ | Hui Lan | |
AT3G09600_AT1G49720_0.html correct. | |||
2020-05-05 | update_netowrk_by_force.py: curl http://118.25.96.118/brain/before should be ↵ | Hui Lan | |
executed as a cron job. | |||
2020-05-05 | update_netowrk_by_force.py: check why merge_edges.py does not fully work. | Hui Lan | |
2020-05-04 | merge_edges.py: Remove 'May take a while.' Actually, after insertion speed ↵ | Hui Lan | |
improvement, writting one million rows to a sqlite db table is really fast. | |||
2020-05-04 | merge_edges.py: Remove colon after [merge_edges.py]. | Hui Lan | |
2020-05-04 | merge_edges.py: rename make_new_edge2. Remove 2. | Hui Lan | |
2020-05-04 | merge_edges.py: make sqlite insert really fast. Commit in the end instead ↵ | Hui Lan | |
of commit for each insertion. | |||
2020-05-04 | merge_edges.py: string does not have isnumeric method. | Hui Lan | |
2020-05-03 | download_and_map.py: when download_log.txt contains blank lines, this script ↵ | Hui Lan | |
will stop working. Make it work when the log file contains blank lines. These blank lines usually appear in the end of the log file. | |||
2020-05-03 | configure.py: fewer daily downloads | Hui Lan | |
2020-05-02 | download_and_map.py: lower the requirement of available disk space to make ↵ | Hui Lan | |
downloading and mapping proceed. | |||
2020-03-27 | update_network_by_force.py: the number of %s should be the same as the ↵ | Hui Lan | |
number of arguments. | |||
2020-03-27 | html_network.py: string like 'F3'H' will cause syntax error in JavaScript. ↵ | Hui Lan | |
json.dumps could not fix that. | |||
2020-03-27 | html_network.py: string like 'F3'H' will cause syntax error in JavaScript. | Hui Lan | |
2020-03-27 | update_network_by_force.py: code for build table of regulators and regulatees. | Hui Lan | |
2020-03-26 | html_network.py: don't show intermediate result | Hui Lan | |
2020-03-26 | html_network.py: remove many lines to make it simpler | Hui Lan | |
2020-02-15 | merge_edges.py: remove old function make_new_edge() | Hui Lan | |
2020-02-15 | update_network_by_force.py: enable visiting the web app each time the ↵ | Hui Lan | |
network is updated The Apache web server will be restarted daily to free memory. sudo service apache2 restart | |||
2020-02-15 | merge_edges.py: now db_fname has a place to print | Hui Lan | |
2020-02-15 | merge_edges.py: an integer in a list prevents the join method from working | Hui Lan | |
2020-02-15 | update_network_by_force.py: disable visiting the web app each time when ↵ | Hui Lan | |
running update_network_by_force.py | |||
2020-02-15 | merge_edges.py: a more memory efficient method to compute an edge's net strength | Hui Lan | |
Compute an edge's strength on the fly instead of saving everything and then computing the net strength. The new function make_new_edge2 will replace make_new_edge. | |||
2020-02-15 | merge_edges.py: do not show edge file names in network.log | Hui Lan | |
2020-02-15 | merge_edges.py: datetime.now() dose not work. Should be datetime.datetime.now() | Hui Lan | |
2020-02-14 | merge_edges.py: write edge file names to network.log | Hui Lan | |
When merging many big edge files, the computer may run out of memory. Save the edge files that have been considered thus far and figure out where merging stopped. | |||
2020-02-11 | merge_edges.py: use the most recent update date as the merged edge's date. | Hui Lan | |
2020-02-11 | merge_edges.py: add a few comments for function make_new_edge. | Hui Lan | |
2020-02-11 | merge_edges.py: log more information in network.log. | Hui Lan | |
2020-02-11 | merge_edges.py: log number of edge files scanned. | Hui Lan | |
2020-02-11 | merge_edges.py: what is the purpose of variable d. | Hui Lan | |
2020-02-11 | merge_edges.py: make a better key | Hui Lan | |
Use a combination of target gene ID and tf gene ID as a key. So if we having the following: Target: AT5G09445 AT5G09445 TF: AT1G53910 RAP2.12 Then the key will be "AT5G09445_AT1G53910". Before it was "AT5G09445 AT5G09445 AT1G53910 RAP2.12". This is OK in most cases, as long a gene ID's corresponding gene name is consistent. But if "AT1G53910" has a different gene name, then we will have a DIFFERENT key, which is not what we want. | |||
2020-02-11 | merge_edges.py: why 10? | Hui Lan | |
2020-02-11 | merge_edges.py: consider all files in directory EDGE_POOL whose file name ↵ | Hui Lan | |
starts with 'edegs' | |||
2020-02-06 | backup_files.py: remove the cd command - we do not need to cd to a directory ↵ | Hui Lan | |
to gzip a file inside it. | |||
2020-02-06 | backup_files.py: fix misspelling | Hui Lan | |
2020-01-21 | update_network_by_force.py: update G.pickle and SOURCE_NODES.pickle with ↵ | Hui Lan | |
edges.txt Visit the following link to update two pickle files used the Webapp, G.pickle and SOURCE_NODES.pickle. http://118.25.96.118/brain/before The visit could be done using the command line tool curl, as follows: curl http://118.25.96.118/brain/before -Hui | |||
2020-01-21 | merge_edges.py: write all edge information to an SQLite database file called ↵ | Hui Lan | |
edges.sqlite When I saved a static html page for each edge (e.g., http://118.25.96.118/static/edges/AT1G20910_AT1G30100_0.html), it took 5GB disk space for saving 1 million html pages. Not very disk space efficient. An alternative is to save all edge information in a database table (i.e., edge), and query this database table for a particular edge. The database file edges.sqlite takes less than 200MB for 1 million edges, requiring 10 times smaller space than the static approach. The reason is that we do not have a lot of HTML tags in the database. Quite happy about that, though it seems that filling a database is a bit slower (2 hours??? for 1 million rows). Also updated two files that were affected: update_network.py and update_network_by_force.py. Now instead of copying 1 million static html page to the Webapp, I just need to copy edges.sqlite to static/edges/. Faster. In the Webapp, I updated start_webapp.py and added a file templates/edge.html for handling dynamic page generation. -Hui | |||
2020-01-18 | download_and_map.py: write "no enough space" warning message to network log file | Hui Lan | |
If there is no enough space left in the disk, download_and_map.py will refuse to download any data. This can be quite mysterious for a maintainer. So, write the reason to the network log file. The reason is something like: "[download_and_map.py] home directory does not have enough space (only 13 G available)." -Hui | |||
2019-12-26 | merge_edges.py: save memory by removing the dictionary variable duniq | Hui Lan | |
The purpose of duniq is to avoid duplicated edge lines. Now, just make sure we don't insert the same tuple. -Hui | |||
2019-12-12 | update_network_by_force.py: summarize the association strength in the edge file | Hui Lan | |
It would be interesting to see how edges' association strengths change over time, as time is an input variable for the function that computes the association strength. | |||
2019-12-07 | create backup_files.py for backing up files | Hui Lan | |
Define a function copy_and_backup_file(src_file, dest_dir) to do backup and compression work. The function copy_and_backup_file is used in update_network_by_force.py. -Hui | |||
2019-12-06 | draw_subnetwork.py: add a known network (G0) from the thermomorphogenesis paper. | Hui Lan | |
Fixed a bug. Now I close figure (plt.close()) before creating a new one, to avoid that the current figure is drawn on top of the old one. -Hui | |||
2019-12-05 | draw_subnetwork.py for generating a subnetwork consisting of genes that ↵ | Hui Lan | |
might be responsible for thermomorphogenesis Use networkx and matplotlib. Reference: Quint et al. (2016) Molecular and genetic control of plant thermomorphogenesis. Nature Plants. |