Github/stackdump
1
0
Fork 0
mirror of https://github.com/djohnlewis/stackdump synced 2025-01-22 14:41:39 +00:00
Code Issues Projects Releases Wiki Activity
130 Commits 5 Branches 5 Tags 92 MiB
bc5642af71
Commit Graph

130 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sam
bc5642af71 Removed the Solr log file. 
Luckily there was nothing sensitive in there... I don't think.
 2013-10-22 20:49:19 +11:00
Sam
09edf16128 Added missing rule to render external links in comments differently. 2013-10-22 08:45:07 +11:00
Sam
436b86b234 Upgrade Apache Solr to 4.5.0 and PySolr to 3.1.0. 
All Solr indexes will need to be re-created.
 2013-10-22 08:44:37 +11:00
Sam
e88e6a31a0 Added a comment about configuring SELinux to allow JRE 7 to run. 2013-10-14 07:44:10 +11:00
Sam
706fd5ef9d Fixed a bug where site names with non-ASCII characters caused a crash. 2013-10-14 07:32:45 +11:00
Sam
9cac41929b Added quotes in a bunch of places so things work with paths containing spaces. 2013-10-14 07:31:50 +11:00
Sam
3615a72310 Removed the -server arg for starting Solr. 
This causes issues with JVMs that don't have the 'server' VM built-in,
e.g. the Windows i386 one. The JVM automatically selects the server
VM anyway when the host characteristics are enough anyway.
 2013-10-14 07:24:10 +11:00
Sam
a472517736 Refactored the import_site command; now caters for filenames of different case (the case changed in 2013 dumps). 2013-09-24 18:07:55 +10:00
Samuel Lai
70fa72b04e Added new external components to README. 2012-12-15 22:53:21 +11:00
Samuel Lai
b667ea0165 Added Mathjax support for math.stackexchange.com. 
Equations and expressions are only rendered in question view at the moment.
 2012-12-15 22:47:46 +11:00
Samuel Lai
993bee4fc1 Added markdown parsing for comments so links in comments now appear properly. 
Also rewrote part of the HTML rewriting code so it doesn't introduce an additional wrapping element in the output which was added due to a html5lib requirements on input.
 2012-12-15 21:43:06 +11:00
Samuel Lai
5ac8492f38 Improved the README file with details on disk space requirements and configuration. 2012-08-25 17:05:13 +10:00
Samuel Lai
36a605711e Added StackExchange question and user URLs to pages as a tooltip to comply with attribution requirements. 
Attribution requirements actually state that hyperlinked URLs should be used, but they would be rather useless in this app so this is an alternative.
 2012-08-25 16:57:01 +10:00
Samuel Lai
e4b2ee80a0 Fixed a bug where extra html/head/body tags were added for every question and answer due to the HTML re-writing process. 2012-08-25 16:37:03 +10:00
Samuel Lai
af28d3e403 Added in a setting to control the number of random questions to show on the search pages. 2012-08-25 16:20:57 +10:00
Samuel Lai
c240356a7b Added a check and a nicer message for when the management commands can't connect to solr. 2012-08-24 18:48:17 +10:00
Samuel Lai
fb38b02758 Added tag v1.0 for changeset 3684617407bb 2012-08-24 18:17:56 +10:00
Samuel Lai
96b1e49311 Added some missing libraries/projects to the credits list in the README. 2012-08-19 13:33:56 +10:00
Samuel Lai
1f9546e4b9 Made some minor amendments to the instructions in the README. 2012-08-19 12:53:42 +10:00
Samuel Lai
049e857159 Handled another exception that may occur if no data has been imported. 2012-08-19 12:47:42 +10:00
Samuel Lai
16e5530a82 Modified download_site_info script to create the data directory if it doesn't exist. 2012-08-19 12:30:33 +10:00
Samuel Lai
c1ae870e3d Startup scripts now create the data directory if it doesn't exist. 2012-08-19 12:27:45 +10:00
Samuel Lai
651f97255e More rendering fixes to README. 2012-08-19 12:15:35 +10:00
Samuel Lai
527d5deb05 Fixed some minor bugs with README and it being rendered by bitbucket. 2012-08-19 12:13:06 +10:00
Samuel Lai
1e6718d850 Merged the cpython-only branch into the default branch. 
The cPython will be the default version; not really much need for the Jython version anymore.
 2012-08-19 11:49:38 +10:00
Samuel Lai
bffe0fd8f5 Added a README. 2012-08-19 11:22:41 +10:00
Samuel Lai
d5bd74feae Changed the default user for the init.d scripts to an arbitrary 'stackdump' user. 2012-08-19 09:50:09 +10:00
Samuel Lai
1b27784a8c Added an error page for when Stackdump fails to connect to Solr. 
Also unified the error pages and added a generic 500 error page.
 2012-08-19 00:09:35 +10:00
Samuel Lai
e0c96a5c5f Fixed a minor styling issue with question titles on search result pages. 2012-08-18 23:38:15 +10:00
Samuel Lai
f25f25019c Created init.d scripts for the Solr and web apps. Compatible with RHEL5. 2012-08-18 23:36:13 +10:00
Samuel Lai
01b0dcae39 Fixed a minor CSS spacing issue between the 'show more comments' block and the moderation message. 2012-08-18 21:40:29 +10:00
Samuel Lai
c1a5382622 Modified app to use a settings file. 
This allows users to change the Solr URL and other things in one spot.
 2012-08-18 21:39:17 +10:00
Samuel Lai
46100e7f01 Fixed a small bug where the 'serving media from' message was printed twice. 2012-08-18 21:08:28 +10:00
Samuel Lai
f4940cd1af Fixed a bug in manage.sh where quoted arguments were not passed on with quotes. 2012-08-18 21:01:27 +10:00
Samuel Lai
4430997467 Amended import instructions to account for the command changes in previous commit. 2012-08-18 21:00:58 +10:00
Samuel Lai
e8adaa9b54 Renamed the commands directory and added a script to make them easier to call. 
Also deleted the get_sites script as it wasn't very useful, and renamed others
to be more meaningful.
 2012-08-18 20:50:13 +10:00
Samuel Lai
e776e95d84 Added an alias for questions for StackExchange style URLs. 
This means site_key/questions/question_id is redirected to site_key/question_id.
 2012-08-18 20:23:14 +10:00
Samuel Lai
5fc56e4329 Added check to ensure the accepted answer to a question actually exists. 
There may be times when it does not exist, e.g. when a question has been merged.
 2012-08-18 20:18:26 +10:00
Samuel Lai
9b9b71077c Added informative message when Stackdump is disabled during a site import. 2012-08-18 20:17:15 +10:00
Samuel Lai
2954dd47ba Added a message for questions that have been closed. 2012-08-18 18:40:18 +10:00
Samuel Lai
6181d83cf3 Added a confirmation prompt when importing so the user can confirm site details. 2012-08-18 18:17:00 +10:00
Samuel Lai
9bcac3f92a Fixed some grammar errors in the footer text. 2012-08-18 17:48:55 +10:00
Samuel Lai
ad5f11260a Changed wording from 'posts' to 'questions' on search results pages. 2012-08-18 17:44:57 +10:00
Samuel Lai
827445105b Excess comments (defaults to any over 3) are now hidden by default. 
They can be shown by clicking on the 'show comments' link.
 2012-08-18 17:44:01 +10:00
Samuel Lai
3d515f51b1 For results with lots of pages, only a limited set of page numbers are rendered. 2012-08-12 16:32:42 +10:00
Samuel Lai
3944261eef Fixed a bug where uncommitted entries from a previously failed import were committed in a later, successful import. 2012-08-12 16:31:30 +10:00
Samuel Lai
1f29fd9113 Modified import.py so it no longer relies on readme.txt. 
readme.txt files were dropped as of the August 2012 data dump.
 2012-08-12 15:40:48 +10:00
Samuel Lai
dd24d98b39 Upgraded Bottle.py to 0.10.11 and CherryPy to 3.2.2. 2012-08-12 14:57:25 +10:00
Samuel Lai
6156d69af0 Further adjusted start_solr.sh for optimal performance. 2012-08-12 14:17:02 +10:00
Samuel Lai
26b803e119 Improved import speed by ~9-fold by actually committing every 1000 questions. 
There was an error made where although questions were only checked for completion every 1000 rows, each completed question was committed separately, resulting in far too many solr calls.

Also modified process to only commit entries in solr at the end, after the database transaction is committed. This means if the process is aborted mid-way through, there won't be orphaned data in solr any more.
 2012-08-12 14:13:15 +10:00