Explore the JFK Assassination files using the new Cognitive Search features of Azure Search. You can watch the demo in action in a short online video or explore the JFK files yourself with our online demo.
This project demonstrates how you can use both the built-in and custom AI in Cognitive Search. Cognitive Search ingests your data from almost any datasource and enriches it using a set of cognitive skills that extracts knowledge and then lets you explore the data using Search.
The JFK files example leverages the built-in Cognitive Skills inside of Azure Search and combines it with custom skills using extensibility. The architecture below showcases how the new Cognitive Search capabilities of Azure enable you to easily create structure from almost any datasource.
Note: This diagram of visuals are inspired by the CIA's JFK document management system in 1997 included in the JFK files.
This project includes the following capabilities for you to build your own version of the JFK files.
- We have provided a subset of the JFK PDF documents and images that have been uploaded to the cloud into Azure Blob Storage.
- An Azure Search service is used to index the content and power the UX experience. We use the new Cognitive Search capabilities to apply pre-built cognitive skills to the content, and we also use the extensibility mechanism to add custom skills using Azure Functions.
- Uses the Cognitive Services Vision API to extract text information from the image via OCR, handwriting, and image captioning,
- Applies Named Entity Recognitition to extract named entities from the documents,
- Annotates text using a custom CIA Cryptonyms skill,
- Generates HOCR content based on results.
- A standalone website to search the index and explore the documents
- This is a demo to showcase a Cognitive Search use case. It is not intended to be a framework or scalable architecture for all scenarios, though it can give you an idea of what your scenario might end up looking like.
- The OCR technology is not perfect and the handwriting capability is in preview. The results will vary greatly by scan and image quality.
- Most file formats and datasources are supported, however some scanned and native PDF formats may not be parsed correctly.
- Cognitive Search is currently only available in public preview in the South Central US and West Europe Azure regions, so the Azure Search service you use with this demo must be provisioned in one of these two regions.
- The Redaction Classifier skill is not currently included in this sample, we'll be publishing an update soon including this skill and instructions on how to deploy your own Redaction Classifier using Azure ML
- IMPORTANT: The JFK Files sample creates a public website and a publicly readable storage container for any extracted images. As-is, it is not suitable for using with non-public data.
These instructions will help you have your own version of the JFK files demo running in Azure in about 20 minutes, with most of that time being provisioning/deployment time.
- An Azure Subscription you can access and deploy resources to.
- Note that this demo does not work properly on the free tier of these resources, and you will incur charges for resources provisioned during this demo for as long as they are provisioned. For an estimation of cost, reference the Azure Pricing Calculator.
- Visual Studio 2017 with Azure Developer Tools enabled. The free community edition will work fine.
- Basic familiarity with using the Azure Portal and cloning and compiling code from github.
-
Click the below button to upload the provided ARM template to the Azure portal, which is written to automatically deploy and configure the following resources:
- An Azure Search service, default set to Standard S1 tier, and deployed to South Central US (regardless of the resource group location you select).
- An Azure Blob Storage Account, default set to Standard LRS tier.
- An Azure App Service plan, default set to Basic B1 tier.
- An Azure Web App Service, using the plan from # 3.
- An Azure Function instance, using the storage account from # 2 and the plan from # 3. The Azure Function will be prepublished with the code provided in this repository as part of the template deployment.
-
Be sure to select the appropriate subscription to deploy to, and create a new resource group for these resources.
-
Review the rest of the parameters as well as the terms and conditions, select the checkbox for "I agree to the terms and conditions stated above", and press "Purchase".
-
Monitor the status of the deployment by following the link that appears in the Azure portal notifications. It takes about 5 minutes for the resources to be fully provisioned and deployed.
- While you are waiting for the resources to finish provisioning, either git clone or download this repo, and open the provided solution file JfkWebApiSkills/JfkWebApiSkills.sln using Visual Studio 2017.
- In the root of the JfkInitializer project, open the App.config file. You will be copying and pasting some secrets into this file momentarily.
- Make sure that the JfkInitializer project is set as your default project if it isn't already.
-
Once your resources are finished deploying (you should get a notification in the Azure portal), navigate to the Outputs section of the deployment.
-
Copy and paste each of the provided outputs from this interface into their corresponding value location in the App.config file from before. Note that one value will be missing from the outputs, the SearchServiceQueryKey.
-
In order to obtain the SearchServiceQueryKey, navigate back to the Overview section of the deployment and find your newly created Azure Search service in the list (Its type will be Microsoft.Search/searchServices). Select it.
-
Select the Keys section, followed by Manage query keys. Copy and paste the key value shown into the SearchServiceQueryKey value of your App.config file.
-
Now you should have filled in all of the required configurations in the App.config file. Save your changes, and press the start button at the top of Visual Studio in order to run the initializer.
-
It will take about 10 minutes for the appropriate search resources to be created, as well as for the website to be deployed. Once the website is deployed, a URL will appear in the console. Copy and paste this URL into a browser to start interacting with what you have created!
This project will create several things for you. It will deploy an Azure Function using the code provided in the JfkWebApiSkills project, as well as create an Azure Search data source, skillset, synonym map, index, and indexer. It will also create a blob storage container that the images will be reuploaded to, as well as deploys the JFK Files frontend to an Azure web app that you can immediately interact with. Feel free to play with the code to see how all of these things are accomplished so that you can start using Cognitive Search for your data enrichment scenario today.