1
0
mirror of https://github.com/djohnlewis/stackdump synced 2025-12-16 21:03:26 +00:00

49 Commits
v1.0 ... v1.2

Author SHA1 Message Date
Sam
722d4125e7 Added section in README re new PowerShell scripts.
Also fixed formatting and wording.
2013-12-01 03:43:58 +11:00
Sam
ce3eb04270 Updated README with v1.2 changes and SO import stats. 2013-12-01 03:33:40 +11:00
Samuel Lai
9613caa8d1 Changed settings so Solr now only listens on localhost, not all interfaces. 2013-11-29 15:18:55 +11:00
Samuel Lai
2583afeb90 Removed more redundant date/time parsing. 2013-11-29 15:11:32 +11:00
Samuel Lai
522e1ff4f2 Fixed bug in script where the directory change was not reverted when script exited. 2013-11-29 15:06:10 +11:00
Samuel Lai
36eb8d3980 Changed the name of the stackdump schema to something better than 'Example'. 2013-11-29 15:05:31 +11:00
Samuel Lai
a597b2e588 Merge import-perf-improvements branch to default. 2013-11-29 13:01:41 +11:00
Samuel Lai
4a9c4504b3 Updated bad docs. 2013-11-29 12:57:06 +11:00
Samuel Lai
77dd2def42 Oops, forgot to re-instate the comment index during the backout. 2013-11-29 01:42:17 +11:00
Samuel Lai
75a216f5a4 Backed out the comments-batching change.
It was causing weird perf issues and errors. Didn't really seem like it made things faster; if anything, things became slower.
2013-11-29 01:12:09 +11:00
Samuel Lai
bf09e36928 Changed other models to avoid unnecessary date/time parsing.
Added PRAGMA statements for comments table and changed flow so the siteId_postId index is now created after data has been inserted.
2013-11-29 00:18:54 +11:00
Samuel Lai
cdb8d96508 Comments are now committed in batches and using a 'prepared' statement via executemany.
Also fixed a Windows compatibility bug with the new temp comments db and a bug with the webapp now that the Comment model has moved. Dates are also no longer parsed from their ISO form for comments; instead left as strings and parsed by SQLObject internally as needed.
2013-11-28 23:51:53 +11:00
Samuel Lai
5868c8e328 Fixed settings for Windows compatibility. 2013-11-28 22:06:33 +11:00
Samuel Lai
8e3d21f817 Fixed settings for Windows compatibility. 2013-11-28 22:06:33 +11:00
Samuel Lai
2fea457b06 Added PowerShell equivalents to launch and manage Stackdump on Windows. 2013-11-28 21:53:45 +11:00
Samuel Lai
6469691e4b Added PowerShell equivalents to launch and manage Stackdump on Windows. 2013-11-28 21:53:45 +11:00
Samuel Lai
65394ac516 More minor fixes. Really should get Stackdump set-up on my dev machine. 2013-11-28 15:07:05 +11:00
Samuel Lai
bcf1d7c71a Again. Forgot to fix site->siteId rename. 2013-11-28 14:39:25 +11:00
Samuel Lai
d36146ae46 More bugs - forgot to rename uses when renaming Comment.site to siteId 2013-11-28 14:38:21 +11:00
Samuel Lai
e1272ce58a Oops, bug with closing temp_db file handle. 2013-11-28 14:35:24 +11:00
Samuel Lai
bff7e13d83 Comment data used during importing is now stored in a separate database to make it easier to delete them afterwards. 2013-11-28 14:23:55 +11:00
Samuel Lai
c0766de8d4 Skips valid XML character scrubbing if configured for faster performance. 2013-11-28 14:01:00 +11:00
Samuel Lai
644269dd5d Added PyCharm project files to the ignore list. 2013-11-28 13:54:47 +11:00
Sam
6bbf0d7b28 Removed a big duplicate file in Solr. 2013-10-22 23:36:46 +11:00
Sam
71c875437e Added tag v1.1 for changeset 3ad1ff15b528 2013-10-22 23:21:20 +11:00
Sam
e78e70e5ac Updated README for v1.1. 2013-10-22 23:20:45 +11:00
Sam
77c76ea9d4 Grrr, forgot to add a file to the previous changeset.
This adds the template that is used when bad Solr syntax is encountered.
2013-10-22 23:20:23 +11:00
Sam
7dc7b7b5bd Solr syntax errors are now handled nicely.
Fixes #4.
2013-10-22 22:43:57 +11:00
Sam
645b24f370 Question permalinks are now recognised and internally linked.
Fixes #3.
2013-10-22 22:03:09 +11:00
Sam
f067353f62 Added answer permalinks and ability to rewrite internal answer permalinks.
This includes a new field in the Solr schema, so a re-index is required after this changeset.

Fixes #1
2013-10-22 21:59:49 +11:00
Sam
4e924f6bd8 Removed the extracted solr-webapp files from the repository.
The files are automatically extracted on launch from the war file.
2013-10-22 20:56:42 +11:00
Sam
bc5642af71 Removed the Solr log file.
Luckily there was nothing sensitive in there... I don't think.
2013-10-22 20:49:19 +11:00
Sam
09edf16128 Added missing rule to render external links in comments differently. 2013-10-22 08:45:07 +11:00
Sam
436b86b234 Upgrade Apache Solr to 4.5.0 and PySolr to 3.1.0.
All Solr indexes will need to be re-created.
2013-10-22 08:44:37 +11:00
Sam
e88e6a31a0 Added a comment about configuring SELinux to allow JRE 7 to run. 2013-10-14 07:44:10 +11:00
Sam
706fd5ef9d Fixed a bug where site names with non-ASCII characters caused a crash. 2013-10-14 07:32:45 +11:00
Sam
9cac41929b Added quotes in a bunch of places so things work with paths containing spaces. 2013-10-14 07:31:50 +11:00
Sam
3615a72310 Removed the -server arg for starting Solr.
This causes issues with JVMs that don't have the 'server' VM built-in,
e.g. the Windows i386 one. The JVM automatically selects the server
VM anyway when the host characteristics are enough anyway.
2013-10-14 07:24:10 +11:00
Sam
a472517736 Refactored the import_site command; now caters for filenames of different case (the case changed in 2013 dumps). 2013-09-24 18:07:55 +10:00
Samuel Lai
70fa72b04e Added new external components to README. 2012-12-15 22:53:21 +11:00
Samuel Lai
b667ea0165 Added Mathjax support for math.stackexchange.com.
Equations and expressions are only rendered in question view at the moment.
2012-12-15 22:47:46 +11:00
Samuel Lai
993bee4fc1 Added markdown parsing for comments so links in comments now appear properly.
Also rewrote part of the HTML rewriting code so it doesn't introduce an additional wrapping element in the output which was added due to a html5lib requirements on input.
2012-12-15 21:43:06 +11:00
Samuel Lai
5ac8492f38 Improved the README file with details on disk space requirements and configuration. 2012-08-25 17:05:13 +10:00
Samuel Lai
36a605711e Added StackExchange question and user URLs to pages as a tooltip to comply with attribution requirements.
Attribution requirements actually state that hyperlinked URLs should be used, but they would be rather useless in this app so this is an alternative.
2012-08-25 16:57:01 +10:00
Samuel Lai
e4b2ee80a0 Fixed a bug where extra html/head/body tags were added for every question and answer due to the HTML re-writing process. 2012-08-25 16:37:03 +10:00
Samuel Lai
af28d3e403 Added in a setting to control the number of random questions to show on the search pages. 2012-08-25 16:20:57 +10:00
Samuel Lai
c240356a7b Added a check and a nicer message for when the management commands can't connect to solr. 2012-08-24 18:48:17 +10:00
Samuel Lai
fb38b02758 Added tag v1.0 for changeset 3684617407bb 2012-08-24 18:17:56 +10:00
Samuel Lai
96b1e49311 Added some missing libraries/projects to the credits list in the README. 2012-08-19 13:33:56 +10:00
886 changed files with 63760 additions and 15257 deletions

View File

@@ -17,8 +17,11 @@ testsuite/.*$
tutorial/.*$
# Solr/Jetty
^java/solr/server/work/.*
^java/solr/server/solr/data/.*
^java/solr/server/solr-webapp/.*
^java/solr/server/logs/.*
# ignore the downloaded logos
^python/media/images/logos/.*
# PyCharm project files
^.idea/

BIN
List-StackdumpCommands.ps1 Normal file

Binary file not shown.

View File

@@ -18,7 +18,30 @@ Besides that, there are no OS-dependent dependencies and should work on any plat
You will also need "7-zip":http://www.7-zip.org/ to extract the data dump files, but Stackdump does not use it directly so you can perform the extraction on another machine first.
It is recommended to run this on a system with at least 3GB of RAM, particularly if you intend to import StackOverflow into Stackdump. Apache Solr requires a fair bit of memory during the import process.
It is recommended that Stackdump be run on a system with at least 3GB of RAM, particularly if you intend to import StackOverflow into Stackdump. Apache Solr requires a fair bit of memory during the import process. It should also have a fair bit of space available; having at least roughly the space used by the raw, extracted, data dump XML files is a good rule of thumb (note that once imported, the raw data dump XML files are not needed by Stackdump any more).
Finally, Stackdump has been tested and works in the latest browsers (IE9, FF10+, Chrome, Safari). It degrades fairly gracefully in older browsers, although some will have rendering issues, e.g. IE8.
h2. Changes and upgrading to v1.1
Version 1.1 fixes a few bugs, the major one being the inability to import the 2013 data dumps due to changes in the case of the filenames. It also adds a couple of minor features, including support for resolving and rewriting short question and answer permalinks.
Because changes have been made to the search schema and the search indexer has been upgraded (to Solr 4.5), all data will need to be re-indexed. Therefore there is no upgrade path; follow the instructions below to set up Stackdump again. It is recommended to install this new version in a new directory, instead of overwriting the existing one.
h2. Changes and upgrading from v1.1 to v1.2.
The major change in the v1.2 release are improvements to the speed of importing data. There are some other smaller changes, including new PowerShell scripts to start and manage Stackdump on Windows as well as a few bug fixes when running on Windows. The search indexing side of things has not changed, therefore data imported using v1.1 will continue to work in v1.2. _Data from older versions however, needs to be re-indexed. See the above section on upgrading to v1.1 for more details._
h3. Importing the StackOverflow data dump, September 2013
The StackOverflow data dump has grown significantly since I started this project back in 2011. With the improvements in v1.2, on a VM with two cores and 4GB of RAM running CentOS 5.7 on a single, standard hard drive containing spinning pieces of metal,
* it took *84719.565491 seconds* to import it, or *23 hours, 31 minutes and 59.565491 seconds*
* once completed, it used up *20GB* of disk space
* during the import, roughly *30GB* of disk space was needed
* the import process used, at max, around *2GB* of RAM.
In total, the StackOverflow data dump has *15,933,529 posts* (questions and answers), *2,332,403 users* and a very large number of comments.
h2. Setting up
@@ -35,6 +58,12 @@ As long as you have:
To provide a better experience, Stackdump can use the RSS feed content to pre-fill some of the required details during the import process, as well as to display the site logos in the app. Stackdump comes bundled with a script that downloads and places these bits in the right places. If you're in a completely offline environment however, it may be worth running this script on a connected box first.
h3. Windows users
If you're using Windows, you will need to substitute the appropriate PowerShell equivalent command for the Stackdump scripts used below. These equivalent PowerShell scripts are in the Stackdump root directory, alongside their Unix counterparts. The names are roughly the same, with the exception of @manage.sh@, which in PowerShell has been broken up into two scripts, @List-StackdumpCommands.ps1@ and @Run-StackdumpCommand.ps1@.
Remember to set your PowerShell execution policy to at least @RemoteSigned@ first as these scripts are not signed. Use the @Get-ExecutionPolicy@ cmdlet to see the current policy, and @Set-ExecutionPolicy@ to set it. You will need to have administrative privileges to set it.
h3. Extract Stackdump
Stackdump was to be self-contained, so to get it up and running, simply extract the Stackdump download to an appropriate location.
@@ -45,6 +74,11 @@ Next, you should verify that the required Java and Python versions are accessibl
Type @java -version@ and check that it is at least version 1.6.
bq. If you're using Java 7 on Linux and you see an error similar to the following -
@ Error: failed /opt/jre1.7.0_40/lib/i386/server/libjvm.so, because /opt/jre1.7.0_40/lib/i386/server/libjvm.so: cannot restore segment prot after reloc: Permission denied @
this is because you have SELinux enabled. You will need to tell SELinux to allow Java to run by using the following command as root (amending the path as necessary) -
@chcon -t textrel_shlib_t /opt/jre1.7.0_40/lib/i386/server/libjvm.so@
Then type @python -V@ and check that it is version 2.5 or later (and not Python 3).
If you would rather not put these versions in the PATH (e.g. you don't want to override the default version of Python in your Linux distribution), you can tell Stackdump which Java and/or Python to use explicitly by creating a file named @JAVA_CMD@ or @PYTHON_CMD@ respectively in the Stackdump root directory, and placing the path to the executable in there.
@@ -98,6 +132,8 @@ To start Stackdump, execute the following command -
If you need to change the port that it runs on, modify @stackdump_dir/python/src/stackdump/settings.py@ and restart the app.
The aforementioned @settings.py@ file also contains some other settings that control how Stackdump works.
Stackdump comes bundled with some init.d scripts as well which were tested on CentOS 5. These are located in the @init.d@ directory. To use these, you will need to modify them to specify the path to the Stackdump root directory and the user to run under.
Both the search indexer and the app need to be running for Stackdump to work.
@@ -118,6 +154,7 @@ h2. Credits
Stackdump leverages several open-source projects to do various things, including -
* "twitter-bootstrap":http://github.com/twitter/bootstrap for the UI
* "jQuery":http://jquery.com for the UI
* "bottle.py":http://bottlepy.org for the web framework
* "cherrypy":http://cherrypy.org for the built-in web server
* "pysolr":https://github.com/toastdriven/pysolr/ to connect from Python to the search indexer, Apache Solr
@@ -125,7 +162,10 @@ Stackdump leverages several open-source projects to do various things, including
* "Jinja2":http://jinja.pocoo.org/ for templating
* "SQLObject":http://www.sqlobject.org/ for writing and reading from the database
* "iso8601":http://pypi.python.org/pypi/iso8601/ for date parsing
* "markdown":http://pypi.python.org/pypi/Markdown for rendering comments
* "mathjax":http://www.mathjax.org/ for displaying mathematical expressions properly
* "httplib2":http://code.google.com/p/httplib2/ as a dependency of pysolr
* "Apache Solr":http://lucene.apache.org/solr/ for search functionality
h2. Things not supported... yet
@@ -136,4 +176,4 @@ h2. Things not supported... yet
h2. License
Stackdump is licensed under the "MIT License":http://en.wikipedia.org/wiki/MIT_License.
Stackdump is licensed under the "MIT License":http://en.wikipedia.org/wiki/MIT_License.

BIN
Run-StackdumpCommand.ps1 Normal file

Binary file not shown.

BIN
Start-Python.ps1 Normal file

Binary file not shown.

BIN
Start-Solr.ps1 Normal file

Binary file not shown.

BIN
Start-StackdumpWeb.ps1 Normal file

Binary file not shown.

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,117 +1,120 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Welcome to the Apache Solr project!
-----------------------------------
Solr is the popular, blazing fast open source enterprise search platform
from the Apache Lucene project.
For a complete description of the Solr project, team composition, source
code repositories, and other details, please see the Solr web site at
http://lucene.apache.org/solr
Getting Started
---------------
See the "example" directory for an example Solr setup. A tutorial
using the example setup can be found at
http://lucene.apache.org/solr/tutorial.html
or in in "docs/tutorial.html" in a binary distribution.
Files included in an Apache Solr binary distribution
----------------------------------------------------
example/
A self-contained example Solr instance, complete with a sample
configuration, documents to index, and the Jetty Servlet container.
Please see example/README.txt for information about running this
example.
dist/apache-solr-XX.war
The Apache Solr Application. Deploy this WAR file to any servlet
container to run Apache Solr.
dist/apache-solr-XX.jar
The Apache Solr Libraries. This JAR file is needed to compile
Apache Solr Plugins (see http://wiki.apache.org/solr/SolrPlugins for
more information).
docs/index.html
The contents of the Apache Solr website.
docs/api/index.html
The Apache Solr Javadoc API documentation.
Instructions for Building Apache Solr from Source
-------------------------------------------------
1. Download the J2SE 5.0 JDK (Java Development Kit) or later from http://java.sun.com.
You will need the JDK installed, and the %JAVA_HOME%\bin directory included
on your command path. To test this, issue a "java -version" command from your
shell and verify that the Java version is 5.0 or later.
2. Download the Apache Ant binary distribution (1.7.x, not 1.6.x, not 1.8.x) from http://ant.apache.org.
You will need Ant installed and the %ANT_HOME%\bin directory included on your
command path. To test this, issue a "ant -version" command from your
shell and verify that Ant is available.
3. Download the Apache Solr distribution, linked from the above
web site. Expand the distribution to a folder of your choice, e.g. c:\solr.
Alternately, you can obtain a copy of the latest Apache Solr source code
directly from the Subversion repository:
http://lucene.apache.org/solr/version_control.html
4. Navigate to the "solr" folder and issue an "ant" command to see the available options
for building, testing, and packaging Solr.
NOTE:
To see Solr in action, you may want to use the "ant example" command to build
and package Solr into the example/webapps directory. See also example/README.txt.
Export control
-------------------------------------------------
This distribution includes cryptographic software. The country in
which you currently reside may have restrictions on the import,
possession, use, and/or re-export to another country, of
encryption software. BEFORE using any encryption software, please
check your country's laws, regulations and policies concerning the
import, possession, or use, and re-export of encryption software, to
see if this is permitted. See <http://www.wassenaar.org/> for more
information.
The U.S. Government Department of Commerce, Bureau of Industry and
Security (BIS), has classified this software as Export Commodity
Control Number (ECCN) 5D002.C.1, which includes information security
software using or performing cryptographic functions with asymmetric
algorithms. The form and manner of this Apache Software Foundation
distribution makes it eligible for export under the License Exception
ENC Technology Software Unrestricted (TSU) exception (see the BIS
Export Administration Regulations, Section 740.13) for both object
code and source code.
The following provides more details on the included cryptographic
software:
Apache Solr uses the Apache Tika which uses the Bouncy Castle generic encryption libraries for
extracting text content and metadata from encrypted PDF files.
See http://www.bouncycastle.org/ for more details on Bouncy Castle.
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Welcome to the Apache Solr project!
-----------------------------------
Solr is the popular, blazing fast open source enterprise search platform
from the Apache Lucene project.
For a complete description of the Solr project, team composition, source
code repositories, and other details, please see the Solr web site at
http://lucene.apache.org/solr
Getting Started
---------------
See the "example" directory for an example Solr setup. A tutorial
using the example setup can be found at
http://lucene.apache.org/solr/tutorial.html
or linked from "docs/index.html" in a binary distribution.
Also, there are Solr clients for many programming languages, see
http://wiki.apache.org/solr/IntegratingSolr
Files included in an Apache Solr binary distribution
----------------------------------------------------
example/
A self-contained example Solr instance, complete with a sample
configuration, documents to index, and the Jetty Servlet container.
Please see example/README.txt for information about running this
example.
dist/solr-XX.war
The Apache Solr Application. Deploy this WAR file to any servlet
container to run Apache Solr.
dist/solr-<component>-XX.jar
The Apache Solr libraries. To compile Apache Solr Plugins,
one or more of these will be required. The core library is
required at a minimum. (see http://wiki.apache.org/solr/SolrPlugins
for more information).
docs/index.html
The Apache Solr Javadoc API documentation and Tutorial
Instructions for Building Apache Solr from Source
-------------------------------------------------
1. Download the Java SE 6 JDK (Java Development Kit) or later from http://java.sun.com/
You will need the JDK installed, and the $JAVA_HOME/bin (Windows: %JAVA_HOME%\bin)
folder included on your command path. To test this, issue a "java -version" command
from your shell (command prompt) and verify that the Java version is 1.6 or later.
2. Download the Apache Ant binary distribution (1.8.2+) from
http://ant.apache.org/ You will need Ant installed and the $ANT_HOME/bin (Windows:
%ANT_HOME%\bin) folder included on your command path. To test this, issue a
"ant -version" command from your shell (command prompt) and verify that Ant is
available.
You will also need to install Apache Ivy binary distribution (2.2.0) from
http://ant.apache.org/ivy/ and place ivy-2.2.0.jar file in ~/.ant/lib -- if you skip
this step, the Solr build system will offer to do it for you.
3. Download the Apache Solr distribution, linked from the above web site.
Unzip the distribution to a folder of your choice, e.g. C:\solr or ~/solr
Alternately, you can obtain a copy of the latest Apache Solr source code
directly from the Subversion repository:
http://lucene.apache.org/solr/versioncontrol.html
4. Navigate to the "solr" folder and issue an "ant" command to see the available options
for building, testing, and packaging Solr.
NOTE:
To see Solr in action, you may want to use the "ant example" command to build
and package Solr into the example/webapps directory. See also example/README.txt.
Export control
-------------------------------------------------
This distribution includes cryptographic software. The country in
which you currently reside may have restrictions on the import,
possession, use, and/or re-export to another country, of
encryption software. BEFORE using any encryption software, please
check your country's laws, regulations and policies concerning the
import, possession, or use, and re-export of encryption software, to
see if this is permitted. See <http://www.wassenaar.org/> for more
information.
The U.S. Government Department of Commerce, Bureau of Industry and
Security (BIS), has classified this software as Export Commodity
Control Number (ECCN) 5D002.C.1, which includes information security
software using or performing cryptographic functions with asymmetric
algorithms. The form and manner of this Apache Software Foundation
distribution makes it eligible for export under the License Exception
ENC Technology Software Unrestricted (TSU) exception (see the BIS
Export Administration Regulations, Section 740.13) for both object
code and source code.
The following provides more details on the included cryptographic
software:
Apache Solr uses the Apache Tika which uses the Bouncy Castle generic encryption libraries for
extracting text content and metadata from encrypted PDF files.
See http://www.bouncycastle.org/ for more details on Bouncy Castle.

View File

@@ -0,0 +1,13 @@
# System Requirements
Apache Solr runs of Java 6 or greater. When using Java 7, be sure to
install at least Update 1! With all Java versions it is strongly
recommended to not use experimental `-XX` JVM options. It is also
recommended to always use the latest update version of your Java VM,
because bugs may affect Solr. An overview of known JVM bugs can be
found on http://wiki.apache.org/lucene-java/JavaBugs.
CPU, disk and memory requirements are based on the many choices made in
implementing Solr (document size, number of documents, and number of
hits retrieved to name a few). The benchmarks page has some information
related to performance on particular platforms.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
java/solr/dist/solr-cell-4.5.0.jar vendored Normal file

Binary file not shown.

BIN
java/solr/dist/solr-clustering-4.5.0.jar vendored Normal file

Binary file not shown.

BIN
java/solr/dist/solr-core-4.5.0.jar vendored Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
java/solr/dist/solr-langid-4.5.0.jar vendored Normal file

Binary file not shown.

BIN
java/solr/dist/solr-solrj-4.5.0.jar vendored Normal file

Binary file not shown.

Binary file not shown.

BIN
java/solr/dist/solr-uima-4.5.0.jar vendored Normal file

Binary file not shown.

BIN
java/solr/dist/solr-velocity-4.5.0.jar vendored Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
java/solr/dist/solrj-lib/noggit-0.5.jar vendored Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,6 @@
The Solr test-framework products base classes and utility classes for
writting JUnit tests excercising Solr functionality.
This test framework relies on the lucene components found in in the
./lucene-libs/ directory, as well as the third-party libraries found
in the ./lib directory.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -1,51 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Solr example configuration
--------------------------
To run this example configuration, use
java -jar start.jar
in this directory, and when Solr is started connect to
http://localhost:8983/solr/admin/
To add documents to the index, use the post.sh script in the exampledocs
subdirectory (while Solr is running), for example:
cd exampledocs
sh post.sh *.xml
See also README.txt in the solr subdirectory, and check
http://wiki.apache.org/solr/SolrResources for a list of tutorials and
introductory articles.
NOTE: This Solr example server references certain Solr jars outside of
this server directory for non-core modules with <lib> statements in
solrconfig.xml. If you make a copy of this example server and wish
to use the ExtractingRequestHandler (SolrCell), DataImportHandler (DIH),
UIMA, the clustering component, or other modules in "contrib",
you will need to copy the required jars into solr/lib or update the paths to
the jars in your solrconfig.xml.
By default, start.jar starts Solr in Jetty using the default solr home
directory of "./solr/" -- To run other example configurations, you can
speciy the solr.solr.home system property when starting jetty...
java -Dsolr.solr.home=multicore -jar start.jar
java -Dsolr.solr.home=example-DIH -jar start.jar

View File

@@ -0,0 +1,8 @@
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<Configure class="org.eclipse.jetty.webapp.WebAppContext">
<Set name="contextPath"><SystemProperty name="hostContext" default="/solr"/></Set>
<Set name="war"><SystemProperty name="jetty.home"/>/webapps/solr.war</Set>
<Set name="defaultsDescriptor"><SystemProperty name="jetty.home"/>/etc/webdefault.xml</Set>
<Set name="tempDirectory"><Property name="jetty.home" default="."/>/solr-webapp</Set>
</Configure>

View File

@@ -0,0 +1,37 @@
#!/bin/bash -ex
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
############
# This script shows how the solrtest.keystore file used for solr tests
# and these example configs was generated.
#
# Running this script should only be necessary if the keystore file
# needs to be replaced, which shouldn't be required until sometime around
# the year 4751.
#
# NOTE: the "-ext" option used in the "keytool" command requires that you have
# the java7 version of keytool, but the generated key will work with any
# version of java
echo "### remove old keystore"
rm -f solrtest.keystore
echo "### create keystore and keys"
keytool -keystore solrtest.keystore -storepass "secret" -alias solrtest -keypass "secret" -genkey -keyalg RSA -dname "cn=localhost, ou=SolrTest, o=lucene.apache.org, c=US" -ext "san=ip:127.0.0.1" -validity 999999

View File

@@ -1,227 +1,205 @@
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">
<!-- =============================================================== -->
<!-- Configure the Jetty Server -->
<!-- -->
<!-- Documentation of this file format can be found at: -->
<!-- http://docs.codehaus.org/display/JETTY/jetty.xml -->
<!-- -->
<!-- =============================================================== -->
<Configure id="Server" class="org.mortbay.jetty.Server">
<!-- Increase the maximum POST size to 1 MB to be able to handle large shard requests -->
<Call class="java.lang.System" name="setProperty">
<Arg>org.mortbay.jetty.Request.maxFormContentSize</Arg>
<Arg>1000000</Arg>
</Call>
<!-- =========================================================== -->
<!-- Server Thread Pool -->
<!-- =========================================================== -->
<Set name="ThreadPool">
<New class="org.mortbay.thread.QueuedThreadPool">
<Set name="minThreads">10</Set>
<Set name="maxThreads">10000</Set>
<Set name="lowThreads">20</Set>
</New>
<!-- Optional Java 5 bounded threadpool with job queue
<New class="org.mortbay.thread.concurrent.ThreadPool">
<Set name="corePoolSize">50</Set>
<Set name="maximumPoolSize">50</Set>
</New>
-->
</Set>
<!-- =========================================================== -->
<!-- Set connectors -->
<!-- =========================================================== -->
<!-- One of each type! -->
<!-- =========================================================== -->
<!-- Use this connector for many frequently idle connections
and for threadless continuations.
-->
<!--
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.nio.SelectChannelConnector">
<Set name="host"><SystemProperty name="jetty.host" /></Set>
<Set name="port"><SystemProperty name="jetty.port" default="8983"/></Set>
<Set name="maxIdleTime">30000</Set>
<Set name="Acceptors">2</Set>
<Set name="statsOn">false</Set>
<Set name="confidentialPort">8443</Set>
<Set name="lowResourcesConnections">5000</Set>
<Set name="lowResourcesMaxIdleTime">5000</Set>
</New>
</Arg>
</Call>
-->
<!-- This connector is currently being used for Solr because it
showed better performance than nio.SelectChannelConnector
for typical Solr requests. -->
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.bio.SocketConnector">
<Set name="host"><SystemProperty name="jetty.host" default="localhost" /></Set>
<Set name="port"><SystemProperty name="jetty.port" default="8983"/></Set>
<Set name="maxIdleTime">50000</Set>
<Set name="lowResourceMaxIdleTime">1500</Set>
<Set name="statsOn">false</Set>
</New>
</Arg>
</Call>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- To add a HTTPS SSL listener -->
<!-- see jetty-ssl.xml to add an ssl connector. use -->
<!-- java -jar start.jar etc/jetty.xml etc/jetty-ssl.xml -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- To allow Jetty to be started from xinetd -->
<!-- mixin jetty-xinetd.xml: -->
<!-- java -jar start.jar etc/jetty.xml etc/jetty-xinetd.xml -->
<!-- -->
<!-- See jetty-xinetd.xml for further instructions. -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- =========================================================== -->
<!-- Set up global session ID manager -->
<!-- =========================================================== -->
<!--
<Set name="sessionIdManager">
<New class="org.mortbay.jetty.servlet.HashSessionIdManager">
<Set name="workerName">node1</Set>
</New>
</Set>
-->
<!-- =========================================================== -->
<!-- Set handler Collection Structure -->
<!-- =========================================================== -->
<Set name="handler">
<New id="Handlers" class="org.mortbay.jetty.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.mortbay.jetty.Handler">
<Item>
<New id="Contexts" class="org.mortbay.jetty.handler.ContextHandlerCollection"/>
</Item>
<Item>
<New id="DefaultHandler" class="org.mortbay.jetty.handler.DefaultHandler"/>
</Item>
<Item>
<New id="RequestLog" class="org.mortbay.jetty.handler.RequestLogHandler"/>
</Item>
</Array>
</Set>
</New>
</Set>
<!-- =========================================================== -->
<!-- Configure the context deployer -->
<!-- A context deployer will deploy contexts described in -->
<!-- configuration files discovered in a directory. -->
<!-- The configuration directory can be scanned for hot -->
<!-- deployments at the configured scanInterval. -->
<!-- -->
<!-- This deployer is configured to deploy contexts configured -->
<!-- in the $JETTY_HOME/contexts directory -->
<!-- -->
<!-- =========================================================== -->
<Call name="addLifeCycle">
<Arg>
<New class="org.mortbay.jetty.deployer.ContextDeployer">
<Set name="contexts"><Ref id="Contexts"/></Set>
<Set name="configurationDir"><SystemProperty name="jetty.home" default="."/>/contexts</Set>
<Set name="scanInterval">5</Set>
</New>
</Arg>
</Call>
<!-- =========================================================== -->
<!-- Configure the webapp deployer. -->
<!-- A webapp deployer will deploy standard webapps discovered -->
<!-- in a directory at startup, without the need for additional -->
<!-- configuration files. It does not support hot deploy or -->
<!-- non standard contexts (see ContextDeployer above). -->
<!-- -->
<!-- This deployer is configured to deploy webapps from the -->
<!-- $JETTY_HOME/webapps directory -->
<!-- -->
<!-- Normally only one type of deployer need be used. -->
<!-- -->
<!-- =========================================================== -->
<Call name="addLifeCycle">
<Arg>
<New class="org.mortbay.jetty.deployer.WebAppDeployer">
<Set name="contexts"><Ref id="Contexts"/></Set>
<Set name="webAppDir"><SystemProperty name="jetty.home" default="."/>/webapps</Set>
<Set name="parentLoaderPriority">false</Set>
<Set name="extract">true</Set>
<Set name="allowDuplicates">false</Set>
<Set name="defaultsDescriptor"><SystemProperty name="jetty.home" default="."/>/etc/webdefault.xml</Set>
</New>
</Arg>
</Call>
<!-- =========================================================== -->
<!-- Configure Authentication Realms -->
<!-- Realms may be configured for the entire server here, or -->
<!-- they can be configured for a specific web app in a context -->
<!-- configuration (see $(jetty.home)/contexts/test.xml for an -->
<!-- example). -->
<!-- =========================================================== -->
<!--
<Set name="UserRealms">
<Array type="org.mortbay.jetty.security.UserRealm">
<Item>
<New class="org.mortbay.jetty.security.HashUserRealm">
<Set name="name">Test Realm</Set>
<Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
<Set name="refreshInterval">0</Set>
</New>
</Item>
</Array>
</Set>
-->
<!-- =========================================================== -->
<!-- Configure Request Log -->
<!-- Request logs may be configured for the entire server here, -->
<!-- or they can be configured for a specific web app in a -->
<!-- contexts configuration (see $(jetty.home)/contexts/test.xml -->
<!-- for an example). -->
<!-- =========================================================== -->
<!--
<Ref id="RequestLog">
<Set name="requestLog">
<New id="RequestLogImpl" class="org.mortbay.jetty.NCSARequestLog">
<Set name="filename"><SystemProperty name="jetty.logs" default="./logs"/>/yyyy_mm_dd.request.log</Set>
<Set name="filenameDateFormat">yyyy_MM_dd</Set>
<Set name="retainDays">90</Set>
<Set name="append">true</Set>
<Set name="extended">false</Set>
<Set name="logCookies">false</Set>
<Set name="LogTimeZone">GMT</Set>
</New>
</Set>
</Ref>
-->
<!-- =========================================================== -->
<!-- extra options -->
<!-- =========================================================== -->
<Set name="stopAtShutdown">true</Set>
<Set name="sendServerVersion">false</Set>
<Set name="sendDateHeader">false</Set>
<Set name="gracefulShutdown">1000</Set>
</Configure>
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<!-- =============================================================== -->
<!-- Configure the Jetty Server -->
<!-- -->
<!-- Documentation of this file format can be found at: -->
<!-- http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax -->
<!-- -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!-- =========================================================== -->
<!-- Server Thread Pool -->
<!-- =========================================================== -->
<Set name="ThreadPool">
<!-- Default queued blocking threadpool -->
<New class="org.eclipse.jetty.util.thread.QueuedThreadPool">
<Set name="minThreads">10</Set>
<Set name="maxThreads">10000</Set>
<Set name="detailedDump">false</Set>
</New>
</Set>
<!-- =========================================================== -->
<!-- Set connectors -->
<!-- =========================================================== -->
<!--
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
<Set name="host"><SystemProperty name="jetty.host" /></Set>
<Set name="port"><SystemProperty name="jetty.port" default="8983"/></Set>
<Set name="maxIdleTime">50000</Set>
<Set name="Acceptors">2</Set>
<Set name="statsOn">false</Set>
<Set name="confidentialPort">8443</Set>
<Set name="lowResourcesConnections">5000</Set>
<Set name="lowResourcesMaxIdleTime">5000</Set>
</New>
</Arg>
</Call>
-->
<!-- This connector is currently being used for Solr because it
showed better performance than nio.SelectChannelConnector
for typical Solr requests. -->
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.bio.SocketConnector">
<Call class="java.lang.System" name="setProperty"> <Arg>log4j.configuration</Arg> <Arg>etc/log4j.properties</Arg> </Call>
<Set name="host"><SystemProperty name="jetty.host" /></Set>
<Set name="port"><SystemProperty name="jetty.port" default="8983"/></Set>
<Set name="maxIdleTime">50000</Set>
<Set name="lowResourceMaxIdleTime">1500</Set>
<Set name="statsOn">false</Set>
</New>
</Arg>
</Call>
<!-- if the connector below is uncommented, then jetty will also accept SSL
connections on port 8984, using a self signed certificate and can
optionally require the client to authenticate with a certificate.
(which can be the same as the server certificate_
# Run solr example with SSL on port 8984
java -jar start.jar
#
# Run post.jar so that it trusts the server cert...
java -Djavax.net.ssl.trustStore=../etc/solrtest.keystore -Durl=https://localhost:8984/solr/update -jar post.jar *.xml
# Run solr example with SSL requiring client certs on port 8984
java -Djetty.ssl.clientAuth=true -jar start.jar
#
# Run post.jar so that it trusts the server cert,
# and authenticates with a client cert
java -Djavax.net.ssl.keyStorePassword=secret -Djavax.net.ssl.keyStore=../etc/solrtest.keystore -Djavax.net.ssl.trustStore=../etc/solrtest.keystore -Durl=https://localhost:8984/solr/update -jar post.jar *.xml
-->
<!--
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
<Arg>
<New class="org.eclipse.jetty.http.ssl.SslContextFactory">
<Set name="keyStore"><SystemProperty name="jetty.home" default="."/>/etc/solrtest.keystore</Set>
<Set name="keyStorePassword">secret</Set>
<Set name="needClientAuth"><SystemProperty name="jetty.ssl.clientAuth" default="false"/></Set>
</New>
</Arg>
<Set name="port"><SystemProperty name="jetty.ssl.port" default="8984"/></Set>
<Set name="maxIdleTime">30000</Set>
</New>
</Arg>
</Call>
-->
<!-- =========================================================== -->
<!-- Set handler Collection Structure -->
<!-- =========================================================== -->
<Set name="handler">
<New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.eclipse.jetty.server.Handler">
<Item>
<New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection"/>
</Item>
<Item>
<New id="DefaultHandler" class="org.eclipse.jetty.server.handler.DefaultHandler"/>
</Item>
<Item>
<New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler"/>
</Item>
</Array>
</Set>
</New>
</Set>
<!-- =========================================================== -->
<!-- Configure Request Log -->
<!-- =========================================================== -->
<!--
<Ref id="Handlers">
<Call name="addHandler">
<Arg>
<New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler">
<Set name="requestLog">
<New id="RequestLogImpl" class="org.eclipse.jetty.server.NCSARequestLog">
<Set name="filename">
logs/request.yyyy_mm_dd.log
</Set>
<Set name="filenameDateFormat">yyyy_MM_dd</Set>
<Set name="retainDays">90</Set>
<Set name="append">true</Set>
<Set name="extended">false</Set>
<Set name="logCookies">false</Set>
<Set name="LogTimeZone">UTC</Set>
</New>
</Set>
</New>
</Arg>
</Call>
</Ref>
-->
<!-- =========================================================== -->
<!-- extra options -->
<!-- =========================================================== -->
<Set name="stopAtShutdown">true</Set>
<Set name="sendServerVersion">false</Set>
<Set name="sendDateHeader">false</Set>
<Set name="gracefulShutdown">1000</Set>
<Set name="dumpAfterStart">false</Set>
<Set name="dumpBeforeStop">false</Set>
<Call name="addBean">
<Arg>
<New id="DeploymentManager" class="org.eclipse.jetty.deploy.DeploymentManager">
<Set name="contexts">
<Ref id="Contexts" />
</Set>
<Call name="setContextAttribute">
<Arg>org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern</Arg>
<Arg>.*/servlet-api-[^/]*\.jar$</Arg>
</Call>
<!-- Add a customize step to the deployment lifecycle -->
<!-- uncomment and replace DebugBinding with your extended AppLifeCycle.Binding class
<Call name="insertLifeCycleNode">
<Arg>deployed</Arg>
<Arg>starting</Arg>
<Arg>customise</Arg>
</Call>
<Call name="addLifeCycleBinding">
<Arg>
<New class="org.eclipse.jetty.deploy.bindings.DebugBinding">
<Arg>customise</Arg>
</New>
</Arg>
</Call>
-->
</New>
</Arg>
</Call>
<Ref id="DeploymentManager">
<Call name="addAppProvider">
<Arg>
<New class="org.eclipse.jetty.deploy.providers.ContextProvider">
<Set name="monitoredDirName"><SystemProperty name="jetty.home" default="."/>/contexts</Set>
<Set name="scanInterval">0</Set>
</New>
</Arg>
</Call>
</Ref>
</Configure>

View File

@@ -0,0 +1,38 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# To use this log config, start solr with the following system property:
# -Djava.util.logging.config.file=etc/logging.properties
## Default global logging level:
.level = INFO
## Log every update command (add, delete, commit, ...)
#org.apache.solr.update.processor.LogUpdateProcessor.level = FINE
## Where to log (space separated list).
handlers = java.util.logging.FileHandler
java.util.logging.FileHandler.level = FINE
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
# 1 GB limit per file
java.util.logging.FileHandler.limit = 1073741824
# Log to the logs directory, with log files named solrxxx.log
java.util.logging.FileHandler.pattern = ./logs/solr%u.log

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -1,36 +0,0 @@
==============================================================
Jetty Web Container
Copyright 1995-2009 Mort Bay Consulting Pty Ltd
==============================================================
The Jetty Web Container is Copyright Mort Bay Consulting Pty Ltd
unless otherwise noted. It is licensed under the apache 2.0
license.
The javax.servlet package used by Jetty is copyright
Sun Microsystems, Inc and Apache Software Foundation. It is
distributed under the Common Development and Distribution License.
You can obtain a copy of the license at
https://glassfish.dev.java.net/public/CDDLv1.0.html.
The UnixCrypt.java code ~Implements the one way cryptography used by
Unix systems for simple password protection. Copyright 1996 Aki Yoshida,
modified April 2001 by Iris Van den Broeke, Daniel Deville.
Permission to use, copy, modify and distribute UnixCrypt
for non-commercial or commercial purposes and without fee is
granted provided that the copyright notice appears in all copies.
The default JSP implementation is provided by the Glassfish JSP engine
from project Glassfish http://glassfish.dev.java.net. Copyright 2005
Sun Microsystems, Inc. and portions Copyright Apache Software Foundation.
Some portions of the code are Copyright:
2006 Tim Vernum
1999 Jason Gilbert.
The jboss integration module contains some LGPL code.
The win32 Java Service Wrapper (v3.2.3) is Copyright (c) 1999, 2006
Tanuki Software, Inc. and 2001 Silver Egg Technology. It is
covered by an open license which is viewable at
http://svn.codehaus.org/jetty/jetty/branches/jetty-6.1/extras/win32service/LICENSE.txt

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,24 @@
# Logging level
solr.log=logs/
log4j.rootLogger=INFO, file, CONSOLE
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%-4r [%t] %-5p %c %x \u2013 %m%n
#- size rotation with log cleanup.
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.MaxFileSize=4MB
log4j.appender.file.MaxBackupIndex=9
#- File to log to and log format
log4j.appender.file.File=${solr.log}/solr.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%-5p - %d{yyyy-MM-dd HH:mm:ss.SSS}; %C; %m\n
log4j.logger.org.apache.zookeeper=WARN
log4j.logger.org.apache.hadoop=WARN
# set to INFO to enable infostream log messages
log4j.logger.org.apache.solr.update.LoggingInfoStream=OFF

View File

@@ -1,54 +1,63 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Example "Solr Home" Directory
=============================
This directory is provided as an example of what a "Solr Home" directory
should look like.
It's not strictly necessary that you copy all of the files in this
directory when setting up a new instance of Solr, but it is recommended.
Basic Directory Structure
-------------------------
The Solr Home directory typically contains the following subdirectories...
conf/
This directory is mandatory and must contain your solrconfig.xml
and schema.xml. Any other optional configuration files would also
be kept here.
data/
This directory is the default location where Solr will keep your
index, and is used by the replication scripts for dealing with
snapshots. You can override this location in the solrconfig.xml
and scripts.conf files. Solr will create this directory if it
does not already exist.
lib/
This directory is optional. If it exists, Solr will load any Jars
found in this directory and use them to resolve any "plugins"
specified in your solrconfig.xml or schema.xml (ie: Analyzers,
Request Handlers, etc...). Alternatively you can use the <lib>
syntax in solrconfig.xml to direct Solr to your plugins. See the
example solrconfig.xml file for details.
bin/
This directory is optional. It is the default location used for
keeping the replication scripts.
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Example Solr Home Directory
=============================
This directory is provided as an example of what a "Solr Home" directory
should look like.
It's not strictly necessary that you copy all of the files in this
directory when setting up a new instance of Solr, but it is recommended.
Basic Directory Structure
-------------------------
The Solr Home directory typically contains the following...
* solr.xml *
This is the primary configuration file Solr looks for when starting.
This file specifies the list of "SolrCores" it should load, and high
level configuration options that should be used for all SolrCores.
Please see the comments in ./solr.xml for more details.
If no solr.xml file is found, then Solr assumes that there should be
a single SolrCore named "collection1" and that the "Instance Directory"
for collection1 should be the same as the Solr Home Directory.
* Individual SolrCore Instance Directories *
Although solr.xml can be configured to look for SolrCore Instance Directories
in any path, simple sub-directories of the Solr Home Dir using relative paths
are common for many installations. In this directory you can see the
"./collection1" Instance Directory.
* A Shared 'lib' Directory *
Although solr.xml can be configured with an optional "sharedLib" attribute
that can point to any path, it is common to use a "./lib" sub-directory of the
Solr Home Directory.
* ZooKeeper Files *
When using SolrCloud using the embedded ZooKeeper option for Solr, it is
common to have a "zoo.cfg" file and "zoo_data" directories in the Solr Home
Directory. Please see the SolrCloud wiki page for more details...
https://wiki.apache.org/solr/SolrCloud

File diff suppressed because it is too large Load Diff

View File

@@ -1,2 +0,0 @@
pizza
history

View File

@@ -1,184 +0,0 @@
#macro(param $key)$request.params.get($key)#end
#macro(url_for_solr)/solr#if($request.core.name != "")/$request.core.name#end#end
#macro(url_for_home)#url_for_solr/browse#end
#macro(q)&q=$!{esc.url($params.get('q'))}#end
#macro(fqs $p)#foreach($fq in $p)#if($velocityCount>1)&#{end}fq=$esc.url($fq)#end#end
#macro(debug)#if($request.params.get('debugQuery'))&debugQuery=true#end#end
#macro(boostPrice)#if($request.params.get('bf') == 'price')&bf=price#end#end
#macro(annotate)#if($request.params.get('annotateBrowse'))&annotateBrowse=true#end#end
#macro(annTitle $msg)#if($annotate == true)title="$msg"#end#end
#macro(spatial)#if($request.params.get('sfield'))&sfield=store#end#if($request.params.get('pt'))&pt=$request.params.get('pt')#end#if($request.params.get('d'))&d=$request.params.get('d')#end#end
#macro(qOpts)#set($queryOpts = $request.params.get("queryOpts"))#if($queryOpts && $queryOpts != "")&queryOpts=$queryOpts#end#end
#macro(lensNoQ)?#if($request.params.getParams('fq') and $list.size($request.params.getParams('fq')) > 0)&#fqs($request.params.getParams('fq'))#end#debug#boostPrice#annotate#spatial#qOpts#end
#macro(lens)#lensNoQ#q#end
#macro(url_for_lens)#{url_for_home}#lens#end
#macro(url_for_start $start)#url_for_home#lens&start=$start#end
#macro(url_for_filters $p)#url_for_home?#q#boostPrice#spatial#qOpts#if($list.size($p) > 0)&#fqs($p)#end#debug#end
#macro(url_for_nested_facet_query $field)#url_for_home#lens&fq=$esc.url($field)#end
## TODO: convert to use {!raw f=$field}$value (with escaping of course)
#macro(url_for_facet_filter $field $value)#url_for_home#lens&fq=$esc.url($field):%22$esc.url($value)%22#end
#macro(url_for_facet_date_filter $field $value)#url_for_home#lens&fq=$esc.url($field):$esc.url($value)#end
#macro(url_for_facet_range_filter $field $value)#url_for_home#lens&fq=$esc.url($field):$esc.url($value)#end
#macro(link_to_previous_page $text)
#if($page.current_page_number > 1)
#set($prev_start = $page.start - $page.results_per_page)
<a class="prev-page" href="#url_for_start($prev_start)">$text</a>
#end
#end
#macro(link_to_next_page $text)
#if($page.current_page_number < $page.page_count)
#set($next_start = $page.start + $page.results_per_page)
<a class="next-page" href="#url_for_start($next_start)">$text</a>
#end
#end
#macro(link_to_page $page_number $text)
#if($page_number == $page.current_page_number)
$text
#else
#if($page_number <= $page.page_count)
#set($page_start = $page_number * $page.results_per_page - $page.results_per_page)
<a class="page" href="#url_for_start($page_start)">$text</a>
#end
#end
#end
#macro(display_facet_query $field, $display, $fieldName)
#if($field.size() > 0)
<span class="facet-field">$display</span>
<ul>
#foreach ($facet in $field)
#if ($facet.value > 0)
#set($facetURL = "#url_for_nested_facet_query($facet.key)")
#if ($facetURL != '')
<li><a href="$facetURL">$facet.key</a> ($facet.value)</li>
#end
#end
#end
</ul>
#end
#end
#macro(display_facet_range_date $field, $display, $fieldName)
<span class="facet-field">$display</span>
##Note: even if mincount is 1, you can still get a '0' before & after
##Note: We assume facet.range.include='lower'
<ul>
#if ($field.before && $field.before > 0)
#set($value = "[* TO " + $date.format("yyyy-MM-dd'T'HH:mm:ss'Z'", $field.start) + "-1MILLIS]")
#set($facetURL = "#url_for_facet_date_filter($fieldName, $value)")
<li><a href="$facetURL">Before</a> ($field.before)</li>
#end
#foreach ($facet in $field.counts)
#set($theDate = $date.toDate("yyyy-MM-dd'T'HH:mm:ss'Z'", $facet.key))
#set($value = '["' + $facet.key + '" TO "' + $facet.key + $field.gap + '-1MILLIS"]')
#set($facetURL = "#url_for_facet_date_filter($fieldName, $value)")
#if ($facetURL != '')
<li><a href="$facetURL">$date.format('MMM yyyy', $theDate)</a> ($facet.value)</li>
#end
#end
#if ($field.after && $field.after > 0)
#set($value = "[" + $date.format("yyyy-MM-dd'T'HH:mm:ss'Z'", $field.after) + " TO *]")
#set($facetURL = "#url_for_facet_date_filter($fieldName, $value)")
<li><a href="$facetURL">After</a> ($field.after)</li>
#end
</ul>
#end
#macro(display_facet_range $field, $display, $fieldName, $start, $end, $gap, $before, $after)
<span class="facet-field">$display</span>
<ul>
#if($before && $before != "")
#set($value = "[* TO " + $start + "]")
#set($facetURL = "#url_for_facet_range_filter($fieldName, $value)")
<li><a href="$facetURL">Less than $start</a> ($before)</li>
#end
#foreach ($facet in $field)
#set($rangeEnd = $math.add($facet.key, $gap))
#set($value = "[" + $facet.key + " TO " + $rangeEnd + "]")
#set($facetURL = "#url_for_facet_range_filter($fieldName, $value)")
#if ($facetURL != '')
<li><a href="$facetURL">$facet.key</a> ($facet.value)</li>
#end
#end
#if($end && $end != "")
#set($value = "[" + $end + " TO *]")
#set($facetURL = "#url_for_facet_range_filter($fieldName, $value)")
<li><a href="$facetURL">More than $math.toNumber($end)</a> ($after)</li>
#end
</ul>
#end
## <lst name="facet_pivot">
## <arr name="cat,inStock">
## <lst>
## <str name="field">cat</str>
## <str name="value">electronics</str>
## <int name="count">17</int>
## <arr name="pivot">
## <lst>
## <str name="field">inStock</str>
## <str name="value">true</str>
## <int name="count">13</int>
## </lst>
## <lst>
## <str name="field">inStock</str>
## <str name="value">false</str>
## <int name="count">4</int>
## </lst>
## </arr>
## </lst>
## $pivots is a list of facet_pivot
#macro(display_facet_pivot $pivots, $display)
#if($pivots.size() > 0)
<span class="facet-field">$display</span>
<ul>
#foreach ($pivot in $pivots)
#foreach ($entry in $pivot.value)
<a href="#url_for_facet_filter($entry.field, $entry.value)">$entry.field::$entry.value</a> ($entry.count)
<ul>
#foreach($nest in $entry.pivot)
<a href="#url_for_facet_filter($entry.field, $entry.value)&fq=$esc.url($nest.field):%22$esc.url($nest.value)%22">$nest.field::$nest.value</a> ($nest.count)
#end
</ul>
#end
#end
</ul>
#end
#end
#macro(field $f)
#if($response.response.highlighting.get($docId).get($f).get(0))
$!response.response.highlighting.get($docId).get($f).get(0)
#else
#foreach($v in $doc.getFieldValues($f))
$v
#end
#end
#end

View File

@@ -1,45 +0,0 @@
#set($searcher=$request.searcher)
#set($params=$request.params)
#set($clusters = $response.response.clusters)
#set($mltResults = $response.response.get("moreLikeThis"))
#set($annotate = $params.get("annotateBrowse"))
#parse('query.vm')
#if($response.response.spellcheck.suggestions and $response.response.spellcheck.suggestions.size() > 0)
Did you mean <a href="#url_for_home?q=$esc.url($response.response.spellcheck.suggestions.collation)#if($list.size($request.params.getParams('fq')) > 0)&#fqs($request.params.getParams('fq'))#end#debug">$response.response.spellcheck.suggestions.collation</a>?
#end
<div class="navigators">
#parse("facets.vm")
</div>
<div class="pagination">
#if($response.response.get('grouped'))
<span><span class="results-found">$response.response.get('grouped').size() group(s)</span> found in ${response.responseHeader.QTime} ms</span>
#else<span><span class="results-found">$page.results_found</span> results found in ${response.responseHeader.QTime} ms</span>
Page <span class="page-num">$page.current_page_number</span> of <span
class="page-count">$page.page_count</span>#end
</div>
<div class="results">
#if($response.response.get('grouped'))
#foreach($grouping in $response.response.get('grouped'))
#parse("hitGrouped.vm")
#end
#else
#foreach($doc in $response.results)
#parse("hit.vm")
#end
#end
</div>
<div class="pagination">
#if($response.response.get('grouped'))
#else
#link_to_previous_page("previous")
<span class="results-found">$page.results_found</span> results found.
Page <span class="page-num">$page.current_page_number</span> of <span
class="page-count">$page.page_count</span>
#link_to_next_page("next")
#end
<br/>
</div>

View File

@@ -1,26 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<h2 #annTitle("Clusters generated by Carrot2 using the /clustering RequestHandler")>Clusters</h2>
<div id="clusters">
Run Solr with java -Dsolr.clustering.enabled=true -jar start.jar to see results
</div>
<script type="text/javascript">
$('#clusters').load("#url_for_solr/clustering#lens",
{'wt':'velocity', 'v.template':"clusterResults"});
</script>

View File

@@ -1,29 +0,0 @@
#foreach ($clusters in $response.response.clusters)
#set($labels = $clusters.get('labels'))
#set($docs = $clusters.get('docs'))
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<h3>#foreach ($label in $labels)$label#if( $foreach.hasNext ),#end#end</h3>
<ol>
#foreach ($cluDoc in $docs)
<li><a href="#url_for_home?q=id:$cluDoc">$cluDoc</a></li>
#end
</ol>
#end

View File

@@ -1,42 +0,0 @@
<div class="result-title"><b>#field('name')</b><span class="mlt">#if($params.getBool('mlt', false) == false)<a href="#lensNoQ&q=id:$docId&mlt=true">More Like This</a>#end</span></div>
##do we have a physical store for this product
#set($store = $doc.getFieldValue('store'))
#if($store)<div class="map"><img src="http://maps.google.com/maps/api/staticmap?&zoom=12&size=150x80&maptype=roadmap&markers=$doc.getFieldValue('store')&sensor=false" /><div><small><a target="_map" href="http://maps.google.com/?q=$store&amp;source=embed">Larger Map</a></small></div></div>#end
<div>Price: $!number.currency($doc.getFieldValue('price'))</div>
<div>Features: #field('features')</div>
<div>In Stock: #field('inStock')</div>
<div class="mlt">
#set($mlt = $mltResults.get($docId))
#set($mltOn = $params.getBool('mlt'))
#if($mltOn == true)<div class="field-name">Similar Items</div>#end
#if ($mltOn && $mlt && $mlt.size() > 0)
<ul>
#foreach($mltHit in $mlt)
#set($mltId = $mltHit.getFieldValue('id'))
<li><div><a href="#url_for_home?q=id:$mltId">$mltId</a></div><div><span class="field-name">Name:</span> $mltHit.getFieldValue('name')</div>
<div><span class="field-name">Price:</span> $!number.currency($mltHit.getFieldValue('price')) <span class="field-name">In Stock:</span> $mltHit.getFieldValue('inStock')</div>
</li>
#end
</ul>
#elseif($mltOn && $mlt.size() == 0)
<div>No Similar Items Found</div>
#end
</div>
#if($params.getBool("debugQuery",false))
<a href="#" onclick='jQuery(this).siblings("pre").toggle(); return false;'>toggle explain</a>
<pre style="display:none">$response.getExplainMap().get($doc.getFirstValue('id'))</pre>
<a href="#" onclick='jQuery(this).siblings("pre2").toggle(); return false;'>toggle all fields</a>
<pre2 style="display:none">
#foreach($fieldname in $doc.fieldNames)
<br>
<span class="field-name">$fieldname :</span>
<span>
#foreach($value in $doc.getFieldValues($fieldname))
$value
#end
</span>
#end
</br>
</pre2>
#end

View File

@@ -1,12 +0,0 @@
#if($response.facetFields)
<h2 #annTitle("Facets generated by adding &facet.field= to the request")>Field Facets</h2>
#foreach($field in $response.facetFields)
<span class="facet-field">$field.name</span>
<ul>
#foreach($facet in $field.values)
<li><a href="#url_for_facet_filter($field.name, $facet.name)">$facet.name</a> ($facet.count)</li>
#end
</ul>
#end
#end

View File

@@ -1,3 +0,0 @@
#set($field = $response.response.facet_counts.facet_queries)
<h2 #annTitle("Facets generated by adding &facet.query= to the request")>Query Facets</h2>
#display_facet_query($field, "", "")

View File

@@ -1,30 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<h2 #annTitle("Facets generated by adding &facet.range= to the request")>Range Facets</h2>
#set($field = $response.response.facet_counts.facet_ranges.price.counts)
#set($start = $response.response.facet_counts.facet_ranges.price.start)
#set($end = $response.response.facet_counts.facet_ranges.price.end)
#set($gap = $response.response.facet_counts.facet_ranges.price.gap)
#set($before = $response.response.facet_counts.facet_ranges.price.before)
#set($after = $response.response.facet_counts.facet_ranges.price.after)
##TODO: Make this display the "range", not just the lower value
##TODO: Have a generic way to deal with ranges
#display_facet_range($field, "Price (in $)", "price", $start, $end, $gap, $before, $after)
#set($field = $response.response.facet_counts.facet_ranges.manufacturedate_dt)
#display_facet_range_date($field, "Manufacture Date", "manufacturedate_dt")

View File

@@ -1,7 +0,0 @@
#parse('facet_fields.vm')
#parse('facet_queries.vm')
#parse('facet_ranges.vm')
#parse('cluster.vm')

Some files were not shown because too many files have changed in this diff Show More