Extract Transform

Identifies the column to which to apply the transform. You can specify only one column.

extract col: MyCol on: 'MyString'

Output: Extracts value My String in new column if it is present in MyCol. Otherwise, new column value is blank.

Usage Notes:

extract col: MyCol after: 'Important:'

Output: Extracts value in MyCol that appears after the string Important:. If the after value does not appear in the column, the output value is blank.

A pattern identifier that precedes the value or pattern to match. Define the after parameter value using string literals, regular expressions, or Wrangle .

Usage Notes:

extract col: MyCol at: 2,6

Output: Extracts contents of MyCol that starts at the second character in the column and extends to the sixth character of the column.

Identifies the start and end point of the pattern to interest.

Parameter inputs are in the form of x,y where x and y are positive integers indicating the starting character and ending character, respectively, of the pattern of interest.

Usage Notes:

The at parameter cannot be combined with any of the following: on, after, before, from , to, and quote. See Pattern Clause Position Matching.

extract col: MyCol before: '|'

Output: Extracts contents of MyCol that occur before the pipe character (|). If the before value does not appear in the column, the output value is blank.

A pattern identifier that occurs after the value or pattern to match. Define the pattern using string literals, regular expressions, or Wrangle .

Usage Notes:

extract col: MyCol from: 'go:'

Output: Extracts contents from MyCol that occur after go:, including go:. If the from value does not appear in the column, the output value is blank.

Identifies the pattern that marks the beginning of the value to match. It can be a string literal, Wrangle , or regular expression. The from value is included in the match.

Usage Notes:

extract col: MyCol on: `###ERROR`

Identifies the pattern to match, which can be a string literal, Wrangle , or regular expression.

Usage Notes:

extract col:MyCol from:'note:' to: `{end}`

Output: Extracts from MyCol column all values that begin with note: up to the end of the value.

Identifies the pattern that marks the ending of the value to match. Pattern can be a string literal, Wrangle , or regular expression. The to value is included in the match.

Usage Notes:

extract col: MyCol quote: 'First' after: `{start}%?`

Output: Extracts each cell value from the MyCol column, starting at the second character in the cell, as long as the string First does not appear in the cell.

Can be used to specify a string as a single quoted object. This parameter value can be one or more characters.

Usage Notes:

extract col: MyCol on: 'My String' ignoreCase: true

Output: Extracts the following values if they appear in the MyCol column: My String, my string, My string, etc.

Indicates whether the match should ignore case or not.

Usage Notes:

extract col: MyCol on: 'z' limit: 3

Output: Extracts each instance of the letter z in the MyCol column into a separate column, generating up to 3 new columns.

The limit parameter defines the maximum number of times that a pattern can be matched within a column.

A set of new columns is generated, as defined by the limit parameter. Each matched instance populates a separate column, until there are no more matches or all of the limit-generated new columns are filled.

Usage Notes:

Source:

Transformation:

The following transformation extracts the second word in the above dataset. Content is extracted after the first space and before the next space.

Name

Custom text or pattern

' '

' '

Results:

Source:

The following represents raw log messages extracted from an application. You want to extract the error level for each message: INFO, WARNING, or ERROR.

Transformation:

The text of interest appears after the timestamp and before the message.

app_log

Custom text or pattern

`###Z `

' '

Results:

This example illustrates the different uses of the replacement transformations to replace or extract cell data.

Source:

The following dataset contains contact information that has been gathered by your marketing platform from actions taken by visitors on your website. You must clean up this data and prepare it for use in an analytics platform.

Transformation:

Title column: For example, you first notice that some data is missing. Your analytics platform recognizes the string value, "#MISSING#" as an indicator of a missing value. So, you click the missing values bar in the Title column. Then, you select the Replace suggestion card. Note that the default replacement is a null value, so you click Edit and update it:

Title

if(ismissing([Title]),'#MISSING#',Title)

Request column: In the Request column, you notice that the reg entry should be cleaned up. Add the following transformation, which replaces that value:

Request

`{start}reg{end}`

Registration

The above transformation uses a Wrangle as the expression of the on: parameter. This expression indicates to match from the start of the cell value, the string literal reg, and then the end of the cell value, which matches on complete cell values of reg only.

This transformation works great on the sample, but what happens if the value is Reg with a capital R? That value might not be replaced. To improve the transformation, you can modify the transformation with the following Wrangle in the on parameter, which captures differences in capitalization:

Request

`{start}{[R|r]}eg{end}`

'Registration'

Add the above transformation to your recipe. Then, it occurs to you that all of the values in the Request column should be capitalized in title or proper case:

Request

proper(Request)

Now, all values are capitalized as titles.

Phone column: You might have noticed some issues with the values in the Phone column. In the United States, the prefix 555 is only used for gathering information; these are invalid phone numbers.

In the data grid, you select the first instance of 555 in the column. However, it selects all instances of that pattern, including ones that you don't want to modify. In this case, continue your selection by selecting the similar instance of 555 in the other row. In the suggestion cards, you click the Replace Text or Pattern transformation.

Notice, however, that the default Replace Text or Pattern transformation has also highlighted the second 555 pattern in one instance, which could be a problem in other phone numbers not displayed in the sample. You must modify the selection pattern for this transformation. In the on: parameter below, the Wrangle has been modified to match only the instances of 555 that appear in the second segment in the phone number format:

Phone

`{start}%{3}-555-%*{end}`

'#INVALID#'

true

Note the wildcard construct has been added (%*). While it might be possible to add a pattern that matches on the last four characters exactly (%{4}), that matching pattern would not capture the possibility of a phone number having an extension at the end of it. The above expression does.

LeadId column: You might have noticed that the lead identifier column (LeadId) contains some embedded information: a date value and an identifier for the instance within the day. The following steps can be used to break out this information. The first one creates a separate working column with this information, which allows us to preserve the original, unmodified column:

Single row formula

LeadId

'LeadIdworking'

You can now work off of this column to create your new ones. First, you can use the following replace transformation to remove the leading two characters, which are not required for the new columns:

LeadIdworking

'LE'

''

Notice that the date information is now neatly contained in the first characters of the working column. Use the following to extract these values to a new column:

LeadIdworking

Custom text or pattern

`{start}%{6}`

The new LeadIdworking2 column now contains only the date information. Cleaning up this column requires reformatting the data, retyping it as a Datetime type, and then applying the dateformat function to format it to your satisfaction. These steps are left as a separate exercise.

For now, let's just rename the column:

Manual rename

LeadIdworking1

'LeadIdDate'

In the first working column, you can now remove the date information using the following:

LeadIdworking

`{start}%{6}`

''

You can rename this column to indicate it is a daily identifier:

Manual rename

LeadIdworking

'LeadIdDaily'

Results: