1
0
mirror of https://github.com/djohnlewis/stackdump synced 2025-02-23 13:44:46 +00:00
Samuel Lai 26b803e119 Improved import speed by ~9-fold by actually committing every 1000 questions.
There was an error made where although questions were only checked for completion every 1000 rows, each completed question was committed separately, resulting in far too many solr calls.

Also modified process to only commit entries in solr at the end, after the database transaction is committed. This means if the process is aborted mid-way through, there won't be orphaned data in solr any more.
2012-08-12 14:13:15 +10:00
Description
stackdump by samuel.lai - imported from bitbucket.org/samuel.lai/stackdump
92 MiB
Languages
Python 86.8%
JavaScript 6.4%
Modelica 4.9%
HTML 0.8%
XSLT 0.5%
Other 0.6%