Solr Client

David Winterfeldt

2009


Spring by Example Utils's HttpClientTemplate, HttpClientOxmTemplate, and SolrOxmClient are used for making different Apache Solr client requests. Solr provides an XML based API over HTTP to the Apache Lucene search engine.

[Note]Note

The Solr server needs to be started before running the unit tests. After downloading Solr and changing to it's directory, the example server can be run and in another console window the sample data can be loaded into the server using the commands below.

$ cd example
$ java -jar start.jar

$ cd example/exampledocs
$ java -jar post.jar *.xml            
        

1. Connecting to Solr using SolrOxmClient

Spring Configuration

The context:component-scan loads the CatalogItemMarshaller which marshalls an update request and unmarshalls a search request. The context:property-placeholder loads values for the Solr host and port. The selectUrl bean sets up the URL for a select, which is used by an HttpClientTemplate, but just for debugging the XML of the search result. The solrOxmClient bean just needs the base url for Solr and a marhsaller and unmarshaller.

SolrOxmClientTest-context.xml
                    
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:p="http://www.springframework.org/schema/p"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans 
                           http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/context 
                           http://www.springframework.org/schema/context/spring-context.xsd">
 
    <!-- Loads CatalogItemMarshaller -->
    <context:component-scan base-package="org.springbyexample.enterprise.solr" />
    
    <context:property-placeholder location="org/springbyexample/enterprise/solr/solr.properties"/>

    <!-- Just used for debugging -->
    <bean id="selectUrl" class="java.lang.String">
        <constructor-arg value="http://${solr.host}:${solr.port}/solr/select" />
    </bean>
                  
    <bean id="solrOxmClient" class="org.springbyexample.httpclient.solr.SolrOxmClient"
          p:baseUrl="http://${solr.host}:${solr.port}/solr"
          p:marshaller-ref="catalogItemMarshaller" 
          p:unmarshaller-ref="catalogItemMarshaller" />

</beans>
                    
                

Code Example

The SolrOxmClient supports marshalling searches and unmarshalling updates. Updates and deletes autocommit, but their methods are overridden so a boolean can be passed in to control commits. It also allows for direct calls to commit, rollback, and optimize. Commit and optimize both support creating requests with max segments, wait flush, and wait searcher by using SolrRequestAttributes.

[Note]Note

The statistics for searches, updates, commits, etc. can be checked using Solr Stats.

Search

Simple search passing in a query. It could be any valid Solr query.

Example 1. Excerpt from SolrOxmClientTest.testSearch()

                        
List<CatalogItem> lCatalogItems = client.search(SEARCH_QUERY_PARAM);
                            
                    

Paginated Search

The 'start' & 'rows' indicate what range of the results to return in the Map. The search query is 'electronics' and is passed into the search along with the Map.

The 'indent' value isn't used by the unmarshaller, and shouldn't be set since it will introduce whitespace in the XML results. In this case it is set so a debug request request that logs the unprocessed XML results is easier to read.

Example 2. Excerpt from SolrOxmClientTest.testPaginatedSearch()

                        
Map<String, String> hParams = new HashMap<String, String>();
hParams.put("start", "5");
hParams.put("rows", "5");
hParams.put("indent", "on");

...

List<CatalogItem> lCatalogItems = client.search("electronics", hParams);
                        
                    

Update

Adds or updates any records in the list based on their id. A commit request is sent immediately after the update.

Example 3. Excerpt from SolrOxmClientTest.testUpdate()

                        
List<CatalogItem> lCatalogItems = new ArrayList<CatalogItem>();

CatalogItem item = new CatalogItem();
item.setId(CATALOG_ITEM_ID);
item.setManufacturer(CATALOG_ITEM_MANUFACTURER);
item.setName(CATALOG_ITEM_NAME);
item.setPrice(CATALOG_ITEM_PRICE);
item.setInStock(CATALOG_ITEM_IN_STOCK);
item.setPopularity(expectedPopularity);

lCatalogItems.add(item);

client.update(lCatalogItems);
                        
                    

Rollback

Update is called passing in the list and a boolean value of false indicating not to commit the update. Then commit or rollback can be called manually. In this example rollback is called.

Example 4. Excerpt from SolrOxmClientTest.testRollback()

                        
List<CatalogItem> lCatalogItems = new ArrayList<CatalogItem>();

CatalogItem item = new CatalogItem();
item.setId(CATALOG_ITEM_ID);
item.setManufacturer(CATALOG_ITEM_MANUFACTURER);
item.setName(CATALOG_ITEM_NAME);
item.setPrice(CATALOG_ITEM_PRICE);
item.setInStock(CATALOG_ITEM_IN_STOCK);
item.setPopularity(popularity);

lCatalogItems.add(item);

// update without commit
client.update(lCatalogItems, false);

...

client.rollback();
                        
                    

Delete

This deletes by using a query matching all 'manu' fields with 'Belkin'. A commit is immediately sent after the delete request. There is also a deleteById(String) for deleting specific records based on their id.

Example 5. Excerpt from SolrOxmClientTest.testDelete()

                        
client.deleteByQuery("manu:Belkin");
                        
                    

Optimize

Sends a request to optimize the search engine.

Example 6. Excerpt from SolrOxmClientTest.testOptimize()

                        
client.optimize();
                        
                    

Creating a marshaller/unmarshaller is the most work setting up the SolrOxmClient since it handles the custom marshalling and unmarshalling between the custom JavaBean and the search fields configured in Solr.

Example 7. CatalogItemMarshaller

Implementation of Spring OXM Marshaller and Unmarshaller using dom4j.

                    
@Component
public class CatalogItemMarshaller implements Marshaller, Unmarshaller {
    
    final Logger logger = LoggerFactory.getLogger(CatalogItemMarshaller.class);

    private static final String ADD_ELEMENT_NAME = "add";
    private static final String DOC_ELEMENT_NAME = "doc";
    private static final String FIELD_ELEMENT_NAME = "field";
    private static final String FIELD_ELEMENT_NAME_ATTRIBUTE = "name";

    /**
     * Implementation of <code>Marshaller</code>. 
     */
    @SuppressWarnings("unchecked")
    public void marshal(Object bean, Result result) throws XmlMappingException, IOException {
        List<CatalogItem> lCatalogItems = (List<CatalogItem>) bean;

        OutputStream out = null;
        XMLWriter writer = null;

        if (result instanceof StreamResult) {
            try {
                out = ((StreamResult) result).getOutputStream();

                Document document = DocumentHelper.createDocument();
                Element root = document.addElement(ADD_ELEMENT_NAME);

                for (CatalogItem item : lCatalogItems) {
                    Element doc = root.addElement(DOC_ELEMENT_NAME);

                    doc.addElement(FIELD_ELEMENT_NAME).addAttribute(FIELD_ELEMENT_NAME_ATTRIBUTE, "id")
                        .addText(item.getId());
                    doc.addElement(FIELD_ELEMENT_NAME).addAttribute(FIELD_ELEMENT_NAME_ATTRIBUTE, "manu")
                        .addText(item.getManufacturer());
                    doc.addElement(FIELD_ELEMENT_NAME).addAttribute(FIELD_ELEMENT_NAME_ATTRIBUTE, FIELD_ELEMENT_NAME_ATTRIBUTE)
                        .addText(item.getName());
                    doc.addElement(FIELD_ELEMENT_NAME).addAttribute(FIELD_ELEMENT_NAME_ATTRIBUTE, "price")
                        .addText(new Float(item.getPrice()).toString());
                    doc.addElement(FIELD_ELEMENT_NAME).addAttribute(FIELD_ELEMENT_NAME_ATTRIBUTE, "inStock")
                        .addText(BooleanUtils.toStringTrueFalse(item.isInStock()));
                    doc.addElement(FIELD_ELEMENT_NAME).addAttribute(FIELD_ELEMENT_NAME_ATTRIBUTE, "popularity")
                        .addText(new Integer(item.getPopularity()).toString());
                }

                writer = new XMLWriter(out);

                writer.write(document);
            } finally {
                try { writer.close(); } catch (Exception e) {}
                IOUtils.closeQuietly(out);
            }

        }

        logger.debug("Marshalled bean of size {}.", lCatalogItems.size());
    }


    /**
     * Implementation of <code>Unmarshaller</code>
     */
    @SuppressWarnings("unchecked")
    public Object unmarshal(Source source) throws XmlMappingException, IOException {
        List<CatalogItem> lResults = new ArrayList<CatalogItem>();

        if (source instanceof StreamSource) {
            InputStream in = null;

            try {
                in = ((StreamSource) source).getInputStream();

                SAXReader reader = new SAXReader();
                Document document = reader.read(in);

                List<Node> lNodes = document.selectNodes("//response/result[@name='response']/doc/*");

                CatalogItem item = null;

                // loop over all matching nodes in order, so can create a new bean 
                // instance on the first match and add it to the results on the last
                for (Node node : lNodes) {
                    if (BooleanUtils.toBoolean(node.valueOf("./@name='id'"))) {
                        item = new CatalogItem();
                        
                        item.setId(node.getText());
                    } else if (BooleanUtils.toBoolean(node.valueOf("./@name='inStock'"))) {
                        item.setInStock(BooleanUtils.toBoolean(node.getText()));
                    } else if (BooleanUtils.toBoolean(node.valueOf("./@name='manu'"))) {
                        item.setManufacturer(node.getText());
                    } else if (BooleanUtils.toBoolean(node.valueOf("./@name='name'"))) {
                        item.setName(node.getText());
                    } else if (BooleanUtils.toBoolean(node.valueOf("./@name='popularity'"))) {
                        item.setPopularity(Integer.parseInt(node.getText()));
                    } else if (BooleanUtils.toBoolean(node.valueOf("./@name='price'"))) {
                        item.setPrice(Float.parseFloat(node.getText()));

                        lResults.add(item);
                    }
                }
            } catch (DocumentException e) {
                throw new UnmarshallingFailureException(e.getMessage(), e);
            } finally {
                IOUtils.closeQuietly(in);
            }

            logger.debug("Unmarshalled bean of size {}.", lResults.size());
        }

        return lResults;
    }

    /**
     * Implementation of <code>Marshaller</code>.
     */
    @SuppressWarnings("unchecked")
    public boolean supports(Class clazz) {
        return (clazz.isAssignableFrom(List.class));
    }

}