Enterprise Search Summit 2010 summary - day 1

I am 'back at the ranch' after an excellent trip to New York for the Enterprise Search Summit 2010 which was a lot of fun and very informative.

A big thank you to my friend and colleague Martin White of Intranet Focus Ltd, a true search expert, who got me involved, and to Michelle Manafy, editor of eContent Magazine and conference chair (and her colleagues) for her excellent job organizing everything.

So even though I had to go buy an old fashioned note pad, due to failure of both my Asus Netbook batteries to take a charge, I made copious notes, so this could be a long post........ Actually I won't give too much detail, as I am sure Michelle would rather you sign up for ESS Fall, the autumnal version of the conference being held in Washington D.C. !

I will start with some observations, before diving into the details:

1. Security is a really big deal for enterprise search implementers !
2. There is no "one ring to rule them all" in enterprise search - most attendee's seem to have more than one search engine / application, and they are OK with this
3. Google Search Appliances and SharePoint 2007 are the poor children compared to the products from more 'established' enterprise search technology vendors

I will examine each of these issues as we work through the conference chronologically. I will break my summary into two posts to keep it more readable. I ran a workshop on Monday morning, on making the most of the search facilities of SharePoint 2007. I should have been a double act between Martin and I, but he could not make it due to a family emergency. I had a great audience, who were very participative and happy to share their experiences, and there was some serious expertise in the group, so a great big thanks to you all !

In the afternoon I attended a deep dive into SharePoint 2010 and FAST 2010, and it was a long and very useful workshop provided by:

• Miles Kehoe, President, New Idea Engineering, Inc
• Mark L. Bennett, CTO, Software Engineering, New Idea Engineering, Inc.
• Jeff Fried, Senior Product Manager, Microsoft
• Natalya Voskresenskaya, Co-founder, Arcovis LLC

They have all just co-operated on a new book: Professional Microsoft FAST Search: Customizing, Designing, and Deploying Search for SharePoint 2010 and Internet Servers and they are a thoroughly great set of people.

Tuesday, the first day of the full conference saw excellent keynotes from Dr Marti Hearst from UC Berkley and David White a senior analyst at Aberdeen Group. Marti put her presentation on SlideShare, so I have embedded it below:

Designing Search For Humans

View more presentations from marti_hearst.

What I liked about Marti's message was that emotional response to User Interfaces is important, and that small details matter when your trying to design an "interrupt free engagement". She also made the excellent point that humans are very social beings, and computers are not, so "don't try to personalize search, socialize it". An excellent keynote.

Google's Rajat Mukherjee had a quick 15 minute slot between the keynotes, and although his focus was obviously on Google's products, a very nice point, which seemed to surprise some attendees, was one about organizational or corporate culture. Within Google, everything is considered open and accessible, unless there is a very good reason to lock it down, so by default all content is shared with everyone. When you consider that they do a lot of 'dog fooding' (i.e. using their own products) that makes sense when you think about the features of Google Docs and Wave.

David Whites keynote was based on Aberdeen group survey of enterprise search implementations and presented in a 'best in class' versus, well lets be polite and say "less than optimal". One thing I would like to draw out from this presentation is the following, all the leaders in implementing best in class enterprise search have a dedicated team. Got that ? Let me shout it again, a dedicated team !

First of the breakout sessions for me was Searching the Past, as session on search and Digital Preservation by Christine Maxwell. I have some experience in long term digital preservation while working for the Open University, and with ESA and NASA on the Mars Express / Beagle 2 space missions, so I found it fascinating ! I will simply pass on some links to interesting projects that Christine brought to our attention:

• CHESHIRE - XML search engine
• CHILIAD - "distributed discovery / alert"
• SHAMAN - multivalent digital preservation framework
• The Blue Ribbon Task Force on Sustainable Digital Preservation and Access

My next session was 'Remembering and using the information you have got' by Ayelette Robinson. My favorite soundbite from Ayelette was "better business through better information access" - yep, amen to that sister..... :-) Ayelette is in Knowledge Management for a law firm, where search is seen as a key decision support tool. Her company uses Recommind as their search tool, and is also using Lexis-Nexis tools for auto-classification (generating metadata from the contents of a document, and classifying the document appropriately). They are big fans of the auto-classification and it is working well for them (but my take on this is that lawyers have very structured documents).

Next up was Avi Rappoport of SearchTools.com on enterprise search architectures specifically Federated versus Aggregated search. If you don't know, Federated (also called 'brokered') search sends the queries to existing search engines / indexes, whereas Aggregated search attempts to construct a single large index, even though it maybe indexing multiple content sources / systems. As per usual, there is not one size fits all, and different approaches fit different contexts. You can always drop Avi a line if you want to know more on this one.

'Improving Findability behind the firewall' was next on my agenda, presented by Bob Boeri, and ECM consultant with considerable expertise (!). Bob sees enterprise search as an element of Enterprise Content Management, and noted the importance of standards, for metadata, taxonomy and other elements of content management strategies, and the positive impact they can have on both precision and recall of your search results.

My last session was a three handed affair, so my notes are bit all over the place, but we can break it down to:

• The Tribune Company (as in the news papers) using tools for Ontology Management
• Raytion on search analytics
• Vivisimo on 'information optimization: efficient searching reduces expert training costs"

Wow, so that's the first day written up for you. Just to summarize, many of the attendee's of my workshop and many other presenters touched on the complexity of their search environments, with multiple search engines / products in use, and nearly everyone noted some issues with security - be it 'early binding' / 'late binding' or security filtering against LDAP directories and how to present results to users. More on these topics, and notes from day 2 in the next posting.