Named Entity Recognition

Use the Named Entity Recognition tool to identify entities, like people, places, and things, in text. The tool leverages the named entity recognition capabilities in the spaCy package. You can use the predefined set of entities or your own custom entities. 

Tool Components

The Named Entity Recognition tool has 4 anchors.

• D input anchor: Connect the text data with entities you want to identify.
• E input anchor (optional): Connect the data with the custom entities you want to identify. This data must contain the custom entity names and labels you want to use to train the model.
• D output anchor: Output new columns of data that display information about the entities in your data.
• M output anchor: Output the model object downstream for use with new data. The model object is compatible with the Predict tool.

Default Model Configuration

Configure the Tool

1. Drag the tool onto the canvas.
2. Connect the D input anchor to text data with entities you want to identify.
3. Select the Language of the text data.
4. Select the Column with Text.
5. Run the workflow.

Default English Entity List

• PERSON: Fictional and non-fictional people.
• NORP: Nationality, religion, or political group.
• FAC: Facilities such as buildings, airports, highways, and bridges.
• ORG: Organizations such as companies, agencies, and institutions.
• GPE: Geographical entities such as countries, cities, and states.
• LOC: Non-GPE locations such as mountain ranges, bodies of water, and continents.
• PRODUCT: Products such as vehicles and foods. Excludes services.
• EVENT: Events such as named hurricanes, wars, and sports events.
• WORK_OF_ART: Works of art such as books, songs, and movies.
• LAW: Named documents made into laws.
• LANGUAGE: Named languages.
• DATE: Date entity.
• TIME: Time entity, less than a day.
• PERCENT: Percentage, includes "%" and the word "percent."
• MONEY: Monetary value, includes the unit.
• QUANTITY: Measurements such as height, weight, and distance.
• ORDINAL: Ordinal entities such as first, second, and third.
• CARDINAL: Numerals that don't fall under another numerical category.

You can find the default entity lists for the other languages in the spaCy documentation.

Custom Model Configuration

If you want to use your own custom entities to train the model, select Train with New Entities. Your source content must contain at least 20 instances of each custom entity. Connect your custom entities to the E input anchor.

Custom Entity List Format

You can use the Text Input tool to pass your own custom entities to the E input anchor. The tool uses your entity list to train a new model. The entity list format is as follows with a few examples:

Configure the Tool

1. Drag the tool onto the canvas.
2. Connect the D input anchor to text data with entities you want to identify.
3. Connect the E input anchor to your custom entity list.
4. Select the Language of the text data connected to the D input anchor.
5. Select the Column with Text from the text data connected to the D input anchor.
6. Select Train with New Entities.
7. Select the Column with Entities from the custom entity list connected to the E input anchor.
8. Select the Column with Labels from the custom entity list connected to the E input anchor.
9. Select the box if you want your model to be Case Sensitive.
10. (Optional) Configure the Train Model section. Refer to the following section for details.
11. Run the workflow.

Output

The D output anchor adds 2 columns to the output:

• entities: This column contains a JSON output with a list of entity tags and descriptions.
  • entity: Entity found by the model.
  • label: The entity label.
  • character_index: The index of the 1st character of the word in the body of text. The index starts at 0.
  • word_index: The index of the word in the body of text. The index starts at 0.
  • entity_length: Character length of the entity.
• entity_diagram: This column contains your text with labeled entities and is visible with the Browse tool.

The M output anchor contains a model object. You can save the model object and use it on new data with the Predict tool.

How to Parse JSON Output

To transform the JSON output to tabular data, use a combination of the JSON Parse, Text To Columns, and Cross Tab tools in this example flow:

1. Pass the Named Entity Recognition tool output to the JSON Parse tool input.
2. Select the entities column under JSON Field.
3. Select Output values into single string field.
4. Pass the JSON Parse tool output to the Text To Columns input.
5. Select the JSON name column under Column to split and set Delimiters to a period (.).
6. Select Split to columns and set Number of columns to 3.
7. Pass the Text to Columns tool output to the Cross Tab tool input.
8. Cross Tab tool configuration:
   1. Group data by these values: Select the column name containing your original text data and the second split JSON name column (by default this is JSON_Name2).
   2. Change Column Headers: Select the third split JSON name column (by default this is JSON_Name3).
   3. Values for New Columns: Select the JSON_ValueString.
   4. Method for Aggregating Values: Select Concatenate.
9. Run your workflow. The Cross Tab tool output now contains the tabular form of the Named Entity Recognition tool output.