Hi there,
Thanks for sharing your requirement.
On my experience with Elastic Search - to provide you a better understanding of how this has been accomplished by me - I would like to run you through an example of one of my projects.
This was the "Expert Witness Profiler" project where I have a lot of unstructured and unorganized data in the form of witness files with detailed description of the cases under and post trial. These files had a plethora of information and we had to somehow extract the pieces of information related to a certain kind of profiles and witnesses. I made use of the "River" concept to pull the data and Logstash to pipeline the data. Exploiting Kibana - I channelized the data in the form of blocks in Elasticsearch. Using this mechanism I could break the document information into relevant pieces that helped me to create small, fragmented, structure data items in ElasticSearch to find the names of the witnesses and profiles that were relevant for any custom search.
As our backend for JL was in Mongodb so we use mongoosastic (using elasticsearch with mongodb). We used Kibana for visualizing and analyzing the data query,and provides an interface to end user with all the info.
Kind regards,
Dave