Process Text: Redact, Translate and Score

Open-text answers are full of insight, and sometimes things you’d rather not show. Four wrangle steps help you prepare text before it reaches reports and dashboards: Redaction hides words, Translation puts everything in one language, Text Scoring adds sentiment and other measures, and Focus Words tags comments with topics.

Each step writes its results to new variables (or, for Translation, to a variable you choose), so your original text is kept.

When to use this

Use these steps when comments are going on a dashboard (redact first), when responses come in several languages (translate first), or when you want to chart sentiment or topics alongside your other measures.

Permission Requirements

You need edit access to the wrangle. See Build a wrangle.

All four steps work the same way on the canvas: drag the step from MICROSERVICES, connect it between your Reader and Writer, and click it to open its settings. Remember to add the new variables to the Writer’s Output Fields.

---

Redaction

“The Redact node allows you to specify words to suppress from text.” It’s most often used to remove profanity from comments before a wider audience sees them. Redacted words are replaced with ****.

1. Choose a word list (optional) - Click Redaction Source to pick a data store that holds your list of words, then choose its Select Redaction Variable. Tick IsRegEx if that list contains patterns rather than plain words.
2. Add your own words - Click Add Word to List to add a word, one at a time.
3. Protect words you want kept - Click Words to Ignore to add words that should never be redacted.
4. Check your rules - Click View Rules to see and remove the words you’ve added.
5. Choose the text to redact - Move variables from Available Fields to Selected Fields. Text, Long Text and multi-response variables can be chosen.

You need a Redaction Source, words added with Add Word to List, or both.

Each selected variable gets a new variable with _redacted on the end, for example comment_redacted. Words are matched as whole words, ignoring capitals.

---

Translation

“The Translate node allows you to translate one or more pieces of text into another language.” For example, comments in Spanish, French and German can all become English. Each comment’s original language is detected automatically.

1. Click Choose variable(s) to translate - The Pick Text to Translate dialog opens.
2. Choose the source - Select one Text or Long Text variable, then click NEXT.
3. Choose where the translation goes - New Field (the default): click Add New Variable and name it. Or Existing Field (overwrite): choose a variable to write over.
4. Repeat for other variables - Each pair appears in Translation Variable Pairs.
5. Choose the Target Language - All pairs in the step are translated into this language.

Existing Field (overwrite) replaces values every run

If you translate into an existing field, its values are replaced each time the wrangle runs. Use New Field if you want to keep what’s there.

Translating EVE conversations

If the text is an EVE conversation, tick IsEve so only the participant’s responses are translated. Tick Translate Prompt as well to translate EVE’s questions too.

---

Text Scoring

“The Text Scoring node allows you to perform advanced Text Analytics processes over any body of text that you bring in to your Wrangle.”

1. Choose a service - Pick one from Select Text Scoring Service. Its description appears alongside.
2. Choose the text - Move Text, Long Text or multi-response variables into Selected Fields.
3. Sentiment only: choose the scale - Tick Display sentiment score as number between -100 and 100 for whole-number scores.

Each service adds new variables named after the source, for example comment_compound:

Service	What it does	New variables
Sentiment Scoring	Positive, negative and neutral scores, plus an overall compound score	_pos, _neg, _neu, _compound (numbers)
Parts of Speech Words	The nouns, adjectives and verbs used, separated by commas	_poswords (text)
Stress Words	A score for words that suggest stress	_stressscore (number)
Danger Words	”Danger Words” that may need further attention	_dangerwords (text)
Noun Chunks	Nouns with the words describing them	_nounchunks (text)

Blank comments are skipped. To show text analytics on a dashboard, see Text analytics in Dashboard Publisher.

---

Focus Words

The Focus Words step tags each comment with the topics from a Focus Words model.

1. Choose a model - Pick one from Focus Words Model. The model’s published version is used.
2. Choose the text - Move the comment variables into Selected Fields. Focus Words Sub Topics is always included; it’s where the topics are written.

The output is a multi-response group called Topics (fwtopics), with one Yes/No variable per sub topic, labelled with the topic and sub topic names.

A wrangle can have only one Focus Words step. Focus Words models are built in the Focus Words app.

---

Frequently Asked Questions

What replaces redacted words?

Four asterisks: ****.

Does Redaction change my original text?

No. It writes the redacted text to a new variable ending _redacted.

Do I need to tell Translation what language the text is in?

No. The original language is detected for each comment. You only choose the Target Language.

Why can’t I choose a variable for Translation?

Only Text and Long Text variables can be translated. Multi-response variables can’t.

Why can’t I drag a variable into Text Scoring or Focus Words?

Choose the service or model first. You’ll see “Please choose a Text Scoring function before proceeding.” or “Please choose a Focus Words Model before proceeding.”

Can I use more than one Focus Words step?

No. A wrangle can have only one. Build a second wrangle for another model.

---

Common Issues and Solutions

”Only one Focus Words node in one Wrangle.”

You tried to add a second Focus Words step.

1. Use the one you have - Or build a separate wrangle for another model.

”Please select FocusWords model.” or “Please select at least 1 variable to proceed.”

The Focus Words step isn’t finished. You’ll see this when you publish.

1. Choose a model - Pick one from Focus Words Model.
2. Choose the text - Add at least one comment variable as well as Focus Words Sub Topics.

”Missing source and destination variables.”

The Translation step has no variable pairs.

1. Add a pair - Click Choose variable(s) to translate and pick a source and destination.

”Variable with this type is not allowed. Please pick another variable.”

You picked a multi-response variable to translate.

1. Pick a text variable - Choose a Text or Long Text variable instead.