Github/stackdump
1
0
Fork 0
mirror of https://github.com/djohnlewis/stackdump synced 2024-12-04 23:17:37 +00:00
Code Issues Projects Releases Wiki Activity

Fixed some minor bugs with README and it being rendered by bitbucket.

Browse Source
This commit is contained in:
Samuel Lai 2012-08-19 12:13:06 +10:00
parent 1e6718d850
commit 527d5deb05
1 changed files with 12 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
README.textile
View File

@ -1,14 +1,14 @@
h1. Stackdump - an offline browser for StackExchange sites.
Stackdump was conceived for those who work in work environments that do not allow easy access to the StackExchange family of websites. It allows you to host a read-only instance of the StackExchange sites locally, accessible via a web browser.
Stackdump was conceived for those who work in environments that do not have easy access to the StackExchange family of websites. It allows you to host a read-only instance of the StackExchange sites locally, accessible via a web browser.
Stackdump comprises of two components - the search indexer ("Apache Solr":http://lucene.apache.org/solr/) and the web application. It uses the "StackExchange Data Dumps":http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/, published quarterly by StackExchange, as its source of data.
h2. Screenshots
!http://edgylogic.com/dynmedia/301/640x480/!
!http://edgylogic.com/dynmedia/303/640x480/!
!http://edgylogic.com/dynmedia/302/640x480/!
"Stackdump home":http://edgylogic.com/dynmedia/301/640x480/
"Stackdump search results":http://edgylogic.com/dynmedia/303/640x480/
"Stackdump question view":http://edgylogic.com/dynmedia/302/640x480/
h2. System Requirements
@ -27,9 +27,9 @@ Stackdump was designed for offline environments or environments with poor intern
As long as you have:
* "Python":http://python.org/download/,
* "Java":http://java.com/en/download/manual.jsp,
* Stackdump,
* "Stackdump"https://bitbucket.org/samuel.lai/stackdump/downloads,
* the "StackExchange Data Dump":http://www.clearbits.net/creators/146-stack-exchange-data-dump (Note: this is only available as a torrent), and
* "7-zip" (needed to extract the data dump files)
* "7-zip":http://www.7-zip.org/ (needed to extract the data dump files)
...you should be able to get an instance up and running.
@ -41,13 +41,13 @@ Stackdump was to be self-contained, so to get it up and running, simply extract
h3. Verify dependencies
Next, you should verify that the required Java and Python versions are accessible in the path.
Next, you should verify that the required Java and Python versions are accessible in the PATH.
Type @java -version@ and check that it is at least version 1.6.
Then type @python -V@ and check that it is version 2.5 or later (and not Python 3).
If you would rather not put these versions in the path (e.g. you don't want to override the default version of Python in your Linux distribution), you can tell Stackdump which Java and/or Python to use explicitly by creating a file named @JAVA_CMD@ or @PYTHON_CMD@ respectively in the Stackdump root directory, and placing the path to the executable in there.
If you would rather not put these versions in the PATH (e.g. you don't want to override the default version of Python in your Linux distribution), you can tell Stackdump which Java and/or Python to use explicitly by creating a file named @JAVA_CMD@ or @PYTHON_CMD@ respectively in the Stackdump root directory, and placing the path to the executable in there.
h3. Download additional site information
@ -60,9 +60,9 @@ To start the download, execute the following command in the Stackdump root direc
If Stackdump will be running in a completely offline environment, it is recommended that you extract and run this command in a connected environment first. If that is not possible, you can manually download the required pieces -
* download the "RSS feed":http://stackexchange.com/feeds/sites to a file
* for each site you will be importing, work out the __site key__ and download the logo by substituting the site key into this URL: http://sstatic.net/<strong>site_key</strong>/img/icon-48.png where *site_key* is the site key. The site key is generally the bit in the URL before .stackexchange.com, or just the domain without the TLD, e.g. for the Salesforce StackExchange at http://salesforce.stackexchange.com, it is just __salesforce__, while for Server Fault at http://serverfault.com, it is __serverfault__.
* for each site you will be importing, work out the __site key__ and download the logo by substituting the site key into this URL: http://sstatic.net/site_key/img/icon-48.png where *site_key* is the site key. The site key is generally the bit in the URL before .stackexchange.com, or just the domain without the TLD, e.g. for the Salesforce StackExchange at http://salesforce.stackexchange.com, it is just __salesforce__, while for Server Fault at http://serverfault.com, it is __serverfault__.
The RSS feed file should be copied to the file @stackdump_dir/data/sites@, and the logos should be copied to @stackdump_dir/python/media/images/logos@ and named with the site key and extension, e.g. @serverfault.png@.
The RSS feed file should be copied to the file @stackdump_dir/data/sites@, and the logos should be copied to the @stackdump_dir/python/media/images/logos@ directory and named with the site key and file type extension, e.g. @serverfault.png@.
h3. Import sites
@ -74,7 +74,7 @@ To start the import process, execute the following command -
@stackdump_dir/manage.sh import_site --base-url site_url --dump-date dump_date path_to_xml_files@
... where __site_url__ is the URL of the site you're importing, e.g. __android.stackexchange.com__; __dump_date__ is the date of the data dump you're importing, e.g. __August 2012__, and finally __path_to_xml_files__ is the path to the XML files you just extracted. The __dump_date__ is a text string that is shown in the app only, so it can be in any format you want.
... where site_url is the URL of the site you're importing, e.g. __android.stackexchange.com__; dump_date is the date of the data dump you're importing, e.g. __August 2012__, and finally path_to_xml_files is the path to the XML files you just extracted. The dump_date is a text string that is shown in the app only, so it can be in any format you want.
This can take anywhere between a minute to 10 hours or more depending on the site you're importing. As a rough guide, __android.stackexchange.com__ took a minute on my VM, while __stackoverflow.com__ took just over 10 hours.
@ -108,7 +108,7 @@ Stackdump leverages several open-source projects to do various things, including
* "iso8601":http://pypi.python.org/pypi/iso8601/ for date parsing
* "httplib2":http://code.google.com/p/httplib2/ as a dependency of pysolr
h2. Things not supported
h2. Things not supported... yet
* searching or browsing by tags
* tag wiki pages