WCS SOLR standard deployment Topology

Posted by Unknown on

In this blog we will analyze how WCS sets up SOLR search engine for standard deployment topology.
There are few subtle differences between the way SOLR enablement works in FEP 3 vs FEP5, for all practical purposes I'm working with the latest feature pack.

Please refer to infocenter documentation if you need to perform this procedure on FEP3.

Background

#1 WebSphere Commerce search Web server's httpd.conf file is automatically created during store-enhancements feature enablement.

#2 Existing install of IHS which is used to run httpd.conf of WCS server's http server can be used to spawn another IHS instance with the SOLR httpd.conf that was generated as part of SOLR enablement.
Usually in production environment IHS process runs on dedicated boxes and hence are remote to the box that houses WCS instance,hence after SOLR enablement we should manually copy the generated httpd.conf to the remote IHS servers and spawn a new IHS process for SOLR.

#3 SOLR enablement creates a new WAS application server profile for SOLR application EAR
SOLR WAS Admin Console by default will be configured on port 9062

http://localhost:9062/ibm/console/




Step 1

To enable SOLR we need to first enable "foundation", this can be done by running following command enablement script for "foundation".

Before performing any task ensure you take a backup of your WCS DB schema, This can be used later on in case enablement process errors out half way through.

"store-enhancements" implicitly enables foundation and Management center feature as well, 
you can either enable foundation alone for SOLR search or enable store-enhancements to enable all other features including SOLR.

/apps/wcs/bin/config_ant.sh 
-buildfile /apps/wcs/components/common/xml/enableFeature.xml
-DinstanceName=<DB_INSTANCE>
-DfeatureName=store-enhancements
-DdbUserPassword=<DB_PASSWORD>
-DsolrHome=/apps/wcs/instances/WCSRT/search/solr/home
-DautoConfigSearchWebserver=true
-DisShareWCWebserverProduct=true
-DsearchWebserverHostname=<HOSTNAME>
-DsearchWebserverInstallLocation=/apps/ihs
-DsearchPluginInstallLocation=/apps/ihs/Plugins
-DsearchWebserverInstallLocation=/apps/ihs
-DsearchPluginInstallLocation=/apps/ihs/Plugins

DautoConfigSearchWebserver=true ' and '-DisShareWCWebserverProduct=true' are optional parameters
When we don't pass any parameters for SOLR configuration, by default it will generate the httpd.conf for SOLR in following directory /components/foundation/subcomponents/search/deploy/ihsconf

NOTE: This step will take hours, on my environment it took close to 3 hours for enablement script to complete.

Step 2

Now we need to ensure that the SOLR JVM is started, this can be checked by running a ps command as follows
ps -elf |grep solrServer


Start the IHS process for SOLR server as follows, if store-enhancements was enabled with SOLR parameter

<IHS_HOME>/ihs/bin/httpd -d /apps/ihs -d /apps/ihs -k start -f <WCS_HOME/>/wcs/instances/WCSRT/search/solr/home/httpconf/httpd.conf

Start like this if store-enhancements was enabled without SOLR parameter
<IHS_HOME>/ihs/bin/httpd -d /apps/ihs -d /apps/ihs -k start -f /apps/wcs/components/foundation/subcomponents/search/deploy/ihsconf/WCSRT_solr/httpd.conf

Make sure you are able to logon to WAS Admin console for SOLR instance

Step 3

Logon to WAS Admin console for SOLR instance and ensure Solr App is started, by default this may not be running, you will have to start it from WAS admin console as shown in the screenshot





Stop and Start the server
<WCS_HOME/>/was/AppServer/profiles/WCSRT_solr/bin/stopServer.sh solrServer
<WCS_HOME/>/was/AppServer/profiles/WCSRT_solr/bin/startServer.sh solrServer

Now validate if you are able to access the SOLR test page


You should see a similar output on the browser

<response><lst name="responseHeader"><int name="status">0</int><int name="QTime">91</int><lst name="params"><str name="q">*:*</str></lst></lst><result name="response" numFound="0" start="0"/></response>

At times you may notice following error while accessing SOLR test page, to fix this problem ensure that the SOLR application is started and restart the server instance after restarting the app.

[3/19/13 3:03:04:905 UTC] 00000030 CoreContainer E org.apache.solr.common.SolrException log java.lang.RuntimeException: Can't find resource 'solrconfig.xml' in classpath or 'solr/./conf/', cwd=/apps/was/AppServer/profiles/WCSRT_solr
        at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:268)
        at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:234)
        at org.apache.solr.core.Config.<init>(Config.java:141)
        at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:131)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:435)
        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
        at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:133)

Step 4


For stores which were already published we will have to setup SOLR endex to use store-enhancements feature for WCS search.

All solr setup utilities (setupsearchindex, preprocess and builidnex) use the master catalog ID. 
The sales catalogs are included in the master catalog's index so the index for the sales catalog is build when you build it for the master catalog,you cannot build an index for sales catalog itself.


<WCS_HOME/>/wcs/components/foundation/subcomponents/search/bin/setupSearchIndex.sh -masterCatalogId 10101 -instance WCSRT -dbuser <DB_USER>-dbuserpwd <DB_PASSWORD>

<WCS_HOME/>/wcs/bin/di-preprocess.sh /apps/wcs/instances/WCSRT/search/pre-processConfig/MC_10101/Oracle -instance  WCSRT -dbuser <DB_USER>-dbuserpwd <DB_PASSWORD> -localename en_US

<WCS_HOME/>/wcs/bin/di-buildindex.sh -instance WCSRT -masterCatalogId 10101-dbuser <DB_USER>-dbuserpwd <DB_PASSWORD> -localename en_US

With this you should be able to see SOLR search functonalities enabled on the store front,

All solr setup utilities (setupsearchindex, preprocess and builidnex) use the master catalog ID.                                                                                                                  
The sales catalogs are included in the master catalog's index so the index for the sales catalog is built when you build it for the master  catalog, you cannot build an index for sales catalog itself. If a catentry is part of a sales catalog, it's going to have the sales catalog ID in the catalog ID field in the index:

<arr name="catalog_id">                                              
<long>10001</long>                                                    
<long>10051</long>                                                    
<long>10201</long>                                                    
                                                                     
This list shows all the catalogs this one particular product is part of. Once you build the index for master catalog successfully, the index for sales catalog will be created already. Then you can just run your site
using the sales catalog.

9 comments:

  1. AnonymousApril 5, 2013 at 10:50 AM

    Hi Hari

    I had a problem with solrServer, I'm trying to test the URL localhost:3737/solr/Default/select?q=*%3A* but the answer is "Error 400: Missing solr core name in path".

    When I go at URL localhost:3737/solr/admin/cores I receive a XML file

    00

    Can help you me? Thank you :)

    ReplyDelete
  2. UnknownApril 6, 2013 at 7:34 AM

    Are you trying this on toolkit or server?

    For toolkit you should use following URL http://localhost/solr/Default/select?q=*%3A*

    If this is on server go to WAS Admin console of SOLR WAS instance
    usually the path will be http://:9062/ibm/console/ and check following

    #1 Validate if solr application is running
    #2 Application servers > solrServer > Process definition > Java Virtual Machine > Custom properties
    Validate that the property solr.solr.home is pointing to correct path, E.g.
    wcs/instances/WCSRT/search/solr/home

    ReplyDelete
  3. AnonymousJanuary 8, 2015 at 8:19 AM

    Hello Hari,

    The URL http://localhost:3737/solr/Default/select?q=*%3A* is producing the following error.

    java.lang.RuntimeException: Can't find resource 'solrconfig.xml' in classpath or '/u01/IBM/WebSphere/CommerceServer70/instances/demo/search/solr/home/MC_10001\en_US\CatalogGroup\conf/', cwd=/u01/IBM/WebSphere/Appserver/profiles/demo_solr

    Can you please advise?

    ReplyDelete
  4. AnonymousJanuary 9, 2015 at 1:48 AM

    The above mentioned issue is resolved

    Just so that users know about the resolution:

    Adding solr.solr.home as an environment variable via WAS Admin Console (which internally updates security.xml) solved the issue.

    -Krishna

    ReplyDelete
  5. UnknownApril 20, 2016 at 1:03 AM

    Excellent post and wonderful blog, I really like this type of interesting articles keep it you.live blog

    ReplyDelete
  6. AnonymousSeptember 22, 2017 at 3:34 AM

    I really appreciate information shared above. It’s of great help. If someone want to learn Online (Virtual) instructor lead live training in APACHE SOLR, kindly contact us http://www.maxmunus.com/contact
    MaxMunus Offer World Class Virtual Instructor led training on APACHE SOLR. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.
    For Demo Contact us.
    Saurabh Srivastava
    MaxMunus
    E-mail: saurabh@maxmunus.com
    Skype id: saurabhmaxmunus
    Ph:+91 8553576305 / 080 - 41103383
    http://www.maxmunus.com/


    ReplyDelete
  7. Kangaroo BlogOctober 21, 2021 at 9:40 AM

    Read full detail of Windows 11, All The Features You Need To Know About

    ReplyDelete
  8. Ander SonFebruary 12, 2024 at 1:29 AM

    Hmm interesting perspective, though I think the conclusion may have oversimplified a more nuanced situation. Appreciate the food for thought regardless. Some valid comments made here. There may be more underlying layers to also consider for a fuller picture. Overall a thought-provoking read. tarpaulins uk

    ReplyDelete
  9. Iron ManApril 17, 2024 at 6:00 AM

    I enjoyed reading this article. It was engaging and easy to follow. Looking forward to more content from you.
    Canvas Tarpaulin

    ReplyDelete