Github/stackdump
1
0
Fork 0
mirror of https://github.com/djohnlewis/stackdump synced 2024-12-04 23:17:37 +00:00
Code Issues Projects Releases Wiki Activity
176 Commits 5 Branches 5 Tags 92 MiB
f79df598d3
Commit Graph

176 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Samuel Lai
f79df598d3 Added tag v1.3 for changeset 5c1ae2e2f71a 2014-03-04 15:06:01 +11:00
Samuel Lai
4d6343584a Minor README tweaks. 2014-03-03 17:07:26 +11:00
Samuel Lai
9d1d6b135a Grrr. More textile issues. 2014-02-27 22:02:04 +11:00
Samuel Lai
96b06f7b35 Oops, textile syntax mistake. 2014-02-27 22:00:48 +11:00
Samuel Lai
28d79ea089 Added notes on using supervisor with stackdump. 2014-02-27 21:58:22 +11:00
Samuel Lai
ce7edf1ca0 Minor README tweaks. 2014-02-27 20:44:55 +11:00
Samuel Lai
4254f31859 Updated the README for the next release. 
Fixes #8 by updating the URL to the data dumps.
 2014-02-27 20:39:32 +11:00
Samuel Lai
c11fcfacf6 Fixes #9. Added ability for import_site command to resume importing if the connection to Solr is lost and restored. 2014-02-27 20:12:53 +11:00
Samuel Lai
7764f088c2 Added a setting to disable the rewriting of links and image URLs. 2014-02-27 18:52:25 +11:00
Samuel Lai
a4c6c2c7ba Certain ignored post type IDs are now recognised by the error handler and messages printed as such. 2014-02-27 18:13:04 +11:00
Samuel Lai
01f9b10c27 Fixed #7. Turns out post IDs are not unique across sites. 
This change will require re-indexing of all sites unfortunately. On the upside, more questions to browse!
 2014-02-27 17:57:34 +11:00
Sam
cdb93e6f68 Merged changes. 2014-02-16 01:04:19 +11:00
Sam
0990e00852 Added an original copy of pysolr.py so the custom changes can be worked out. 2014-02-16 01:03:05 +11:00
Samuel Lai
92e359174a Added some notes on importing StackOverflow on Windows. 2013-12-12 17:29:55 +11:00
Samuel Lai
c521fc1627 Added tag v1.2 for changeset 240affa260a1 2013-11-30 18:06:37 +11:00
Sam
722d4125e7 Added section in README re new PowerShell scripts. 
Also fixed formatting and wording.
 2013-12-01 03:43:58 +11:00
Sam
ce3eb04270 Updated README with v1.2 changes and SO import stats. 2013-12-01 03:33:40 +11:00
Samuel Lai
9613caa8d1 Changed settings so Solr now only listens on localhost, not all interfaces. 2013-11-29 15:18:55 +11:00
Samuel Lai
2583afeb90 Removed more redundant date/time parsing. 2013-11-29 15:11:32 +11:00
Samuel Lai
522e1ff4f2 Fixed bug in script where the directory change was not reverted when script exited. 2013-11-29 15:06:10 +11:00
Samuel Lai
36eb8d3980 Changed the name of the stackdump schema to something better than 'Example'. 2013-11-29 15:05:31 +11:00
Samuel Lai
a597b2e588 Merge import-perf-improvements branch to default. 2013-11-29 13:01:41 +11:00
Samuel Lai
4a9c4504b3 Updated bad docs. 2013-11-29 12:57:06 +11:00
Samuel Lai
77dd2def42 Oops, forgot to re-instate the comment index during the backout. 2013-11-29 01:42:17 +11:00
Samuel Lai
75a216f5a4 Backed out the comments-batching change. 
It was causing weird perf issues and errors. Didn't really seem like it made things faster; if anything, things became slower.
 2013-11-29 01:12:09 +11:00
Samuel Lai
bf09e36928 Changed other models to avoid unnecessary date/time parsing. 
Added PRAGMA statements for comments table and changed flow so the siteId_postId index is now created after data has been inserted.
 2013-11-29 00:18:54 +11:00
Samuel Lai
cdb8d96508 Comments are now committed in batches and using a 'prepared' statement via executemany. 
Also fixed a Windows compatibility bug with the new temp comments db and a bug with the webapp now that the Comment model has moved. Dates are also no longer parsed from their ISO form for comments; instead left as strings and parsed by SQLObject internally as needed.
 2013-11-28 23:51:53 +11:00
Samuel Lai
5868c8e328 Fixed settings for Windows compatibility. 2013-11-28 22:06:33 +11:00
Samuel Lai
8e3d21f817 Fixed settings for Windows compatibility. 2013-11-28 22:06:33 +11:00
Samuel Lai
2fea457b06 Added PowerShell equivalents to launch and manage Stackdump on Windows. 2013-11-28 21:53:45 +11:00
Samuel Lai
6469691e4b Added PowerShell equivalents to launch and manage Stackdump on Windows. 2013-11-28 21:53:45 +11:00
Samuel Lai
65394ac516 More minor fixes. Really should get Stackdump set-up on my dev machine. 2013-11-28 15:07:05 +11:00
Samuel Lai
bcf1d7c71a Again. Forgot to fix site->siteId rename. 2013-11-28 14:39:25 +11:00
Samuel Lai
d36146ae46 More bugs - forgot to rename uses when renaming Comment.site to siteId 2013-11-28 14:38:21 +11:00
Samuel Lai
e1272ce58a Oops, bug with closing temp_db file handle. 2013-11-28 14:35:24 +11:00
Samuel Lai
bff7e13d83 Comment data used during importing is now stored in a separate database to make it easier to delete them afterwards. 2013-11-28 14:23:55 +11:00
Samuel Lai
c0766de8d4 Skips valid XML character scrubbing if configured for faster performance. 2013-11-28 14:01:00 +11:00
Samuel Lai
644269dd5d Added PyCharm project files to the ignore list. 2013-11-28 13:54:47 +11:00
Sam
6bbf0d7b28 Removed a big duplicate file in Solr. 2013-10-22 23:36:46 +11:00
Sam
71c875437e Added tag v1.1 for changeset 3ad1ff15b528 2013-10-22 23:21:20 +11:00
Sam
e78e70e5ac Updated README for v1.1. 2013-10-22 23:20:45 +11:00
Sam
77c76ea9d4 Grrr, forgot to add a file to the previous changeset. 
This adds the template that is used when bad Solr syntax is encountered.
 2013-10-22 23:20:23 +11:00
Sam
7dc7b7b5bd Solr syntax errors are now handled nicely. 
Fixes #4.
 2013-10-22 22:43:57 +11:00
Sam
645b24f370 Question permalinks are now recognised and internally linked. 
Fixes #3.
 2013-10-22 22:03:09 +11:00
Sam
f067353f62 Added answer permalinks and ability to rewrite internal answer permalinks. 
This includes a new field in the Solr schema, so a re-index is required after this changeset.

Fixes #1
 2013-10-22 21:59:49 +11:00
Sam
4e924f6bd8 Removed the extracted solr-webapp files from the repository. 
The files are automatically extracted on launch from the war file.
 2013-10-22 20:56:42 +11:00
Sam
bc5642af71 Removed the Solr log file. 
Luckily there was nothing sensitive in there... I don't think.
 2013-10-22 20:49:19 +11:00
Sam
09edf16128 Added missing rule to render external links in comments differently. 2013-10-22 08:45:07 +11:00
Sam
436b86b234 Upgrade Apache Solr to 4.5.0 and PySolr to 3.1.0. 
All Solr indexes will need to be re-created.
 2013-10-22 08:44:37 +11:00
Sam
e88e6a31a0 Added a comment about configuring SELinux to allow JRE 7 to run. 2013-10-14 07:44:10 +11:00