I was invited to speak on 2015.11.10 at the Bay Area Search Meetup in San Jose, CA. With over 175 people marked as attending (and several more on the waitlist who showed up), we had a very exciting and lively discussion for almost 2 hours (about 1/2 was my presentation, with the other half being Q&A mixed in throughout). Thanks again to eBay for hosting the event and providing pizza and beverages, and to everyone who attended for the warm welcome and great discussions.
Video:
Talk Summary:
Search engines frequently miss the mark when it comes to understanding user intent. This talk will walk through some of the key building blocks necessary to turn a search engine into a dynamically-learning “intent engine”, able to interpret and search on meaning, not just keywords. We will walk through CareerBuilder’s semantic search architecture, including semantic autocomplete, query and document interpretation, probabilistic query parsing, automatic taxonomy discovery, keyword disambiguation, and personalization based upon user context/behavior. We will also see how to leverage an inverted index (Lucene/Solr) as a knowledge graph that can be used as a dynamic ontology to extract phrases, understand and weight the semantic relationships between those phrases and known entities, and expand the query to include those additional conceptual relationships.
As an example, most search engines completely miss the mark at parsing a query like (Senior Java Developer Portland, OR Hadoop). We will show how to dynamically understand that “senior” designates an experience level, that “java developer” is a job title related to “software engineering”, that “portland, or” is a city with a specific geographical boundary (as opposed to a keyword followed by a boolean operator), and that “hadoop” is the skill “Apache Hadoop”, which is also related to other terms like “hbase”, “hive”, and “map/reduce”. We will discuss how to train the search engine to parse the query into this intended understanding and how to reflect this understanding to the end user to provide an insightful, augmented search experience.
Topics: Semantic Search, Apache Solr, Finite State Transducers, Probabilistic Query Parsing, Bayes Theorem, Augmented Search, Recommendations, Query Disambiguation, NLP, Knowledge Graphs
@treygrainger gave an excellent talk at #BayAreaSearch #meetup last night. His slides are wonderfully detailed!… https://t.co/lFVLLr9GFy
— Mark Cramer (@markdcramer) November 11, 2015
If you were not at #BayAreaSearchMeetup you missed the great talk #SearchingByIntent by @TreyGrainger the author of #SolrInAction
— Kaamel Kermaani (@kaamel) November 11, 2015
When your hammer is a search engine, you can do everything using queries @treygrainger #bayareasearch
— ts waterman (@tswaterman) November 11, 2015
#eBaySearch Meetup @treygrainger explains implementation of entity-type recognition pic.twitter.com/5hG7gfXwY3
— Irene Rusman (@irusman) November 11, 2015
#eBaySearch #meetup @treygrainger explains the dif between a blind and a semantic query parsing pic.twitter.com/lGji1Vo9wp
— Irene Rusman (@irusman) November 11, 2015
