A Beauty Company's FDE Implementation in Practice: An Eight-Step Pipeline for Cleaning Massive Multimodal Data into Trustworthy Data

A while ago, we went to a beauty company to implement FDE. The specific business included sentiment analysis, competitor research, marketing campaign reviews, and extracting users' genuine feedback on efficacy, skin feel, ingredients, price, and usage scenarios from comment sections.

After the project kicked off, a very practical problem presented itself: data from platforms such as Xiaohongshu, Douyin, WeChat Channels, and Weibo had already been collected, but text, images, audio, and video were all mixed together; the same product had a full name, an abbreviated name, a series name, and users' own ways of referring to it; and original posts, comments, and replies still needed to maintain complete associations. If even one link in the chain was handled incorrectly, the downstream user insights and report conclusions would all go off track.

This article will start from this project and then organize the actual practices into a general solution.

In the future, when there is a need to analyze data from these platforms—whether for consumer goods insights, automotive word-of-mouth, restaurant site selection, travel reviews, or brand sentiment—this pipeline can be referenced to complete data cleaning, relationship organization, quality verification, and storage.

The entire pipeline is divided into eight steps:

Define data specifications → Preserve original content → Map out data relationships → Multimodal normalization → Conflict and quality cleaning → Generate structured data → Perform acceptance checks → Store according to rules and versions

Step 1: Define data specifications first

After entering a project, we always first ask the business team one question: What will this batch of data ultimately be used for?

Sentiment reports need to look at events, spread, and risks; competitor reports need to compare brands, product lines, efficacy, ingredients, and user reviews; marketing reviews need to look at channels, content, interactions, and feedback; and comment section analysis also needs to preserve user questions, context, and reply relationships.

Different uses require different cleaning methods.

For example, the same new product seeding material is published by ten accounts. When doing a spread review, all ten publications need to be retained; when counting organic discussion, they need to be marked as the same group of duplicate content. If nine of them are deleted directly after collection, it will be impossible later to switch the statistical scope.

This is what data specifications are. The name is not complicated, and the purpose is very direct: unify the rules first, then begin processing.

Step 2: Always preserve the original content

The original text, images, audio, video, links, publication time, and platform original IDs must all be retained.

The cleaned text should be stored separately and must not overwrite the original text. OCR, ASR, and model-generated labels can only be treated as newly added results.

If only the model-processed results are retained, then even if errors are discovered later, it will be impossible to know whether the error occurred in collection, recognition, or analysis.

Step 3: Connect users, original posts, comments, and replies

Many teams only save the author, body text, time, and interaction counts. That is barely enough for word frequency statistics, but comment analysis will lose context.

A comment saying "this is way too bad"—is it evaluating the product in the original post, or responding to the previous comment? You have to look at which ID it is attached under.

parent_comment_id: whom this reply is responding to.

The primary key in the database can be written as "platform name + platform original ID" to avoid identical numbers appearing across different platforms.

Do not directly identify people across different platforms. The same nickname or a similar avatar can only be treated as a clue. Without reliable primary keys and explicit authorization, you can aggregate topics, objects, and content, but you must not directly stitch two accounts into one person's complete trajectory.

To summarize in one sentence: clarify relationships within a platform, and store sources separately across platforms.

Step 4: Process text, images, audio, and video separately, then unify the format

The four types of content are processed differently, and in the end all must be organized into one unified format: who said what at what time, what objects were mentioned, and where in the original content the evidence is located.

Text

Preserve the original text, then handle garbled characters, whitespace, links, language, and duplicate paragraphs. When the model extracts opinions, it must also return the corresponding original sentence.

Images

OCR extracts prices, model numbers, dates, and screenshot text; the vision model identifies objects, scenes, and components. Both results must retain the image position so that later it can be determined whether the text and the object belong to the same region.

Audio

First unify the format and sample rate, then perform ASR. The output needs to include time codes. When analyzing multi-person conversations, add speaker diarization; ordinary voice-over does not need an extra layer.

Video

First extract the audio track and keyframes. Run ASR on the audio track, run OCR and visual recognition on the keyframes, and finally merge them according to time.

There is no need to analyze the entire video frame by frame. You can first do low-frequency frame sampling and full-text transcription, and after finding the relevant segments, increase the local frame sampling density.

How to choose between APIs and local models

For small-batch validation, you can directly call APIs.

Once the data volume is stable, move high-frequency tasks to local deployment. A common combination is:

For cloud speech recognition, you can use Tencent Cloud ASR or Volcano Engine audio and video subtitle generation.

Trial quotas and concurrency limits will change, so check the console directly before going live.

When selecting options, do not look only at price. Use your own real samples to test dialects, noise, Chinese-English mixing, vertical-screen subtitles, and small text in screenshots.

When tasks will be executed repeatedly, you can build a lightweight workbench:

File or link enters → automatically select OCR, ASR, or vision tools → view the original text and recognition results → manual correction → export JSON or CSV

The workbench also records call counts, time consumed, costs, and failure reasons. After running for a while, the team will naturally know which steps should continue using APIs and which are worth deploying locally.

Step 5: When data conflicts, look at the evidence first

Multimodal processing often produces two results that both look like correct answers.

We encountered a video: ASR recognized it as 100, while frame OCR recognized it as 11000. If the system simply chooses the side with higher confidence, the wrong number will flow all the way into statistics and reports.

After discovering the conflict, locate the original evidence by timecode and image position, then verify the unit, context, and publication time. If it can be confirmed, correct it; if it cannot be confirmed for now, place it in a pending-verification queue and exclude it from precise numerical statistics.

When correcting, retain the original value, corrected value, evidence level, correction reason, review status, and model version. The original value must not be overwritten.

Step 6: Handle duplicates, low-quality content, and anomalous content separately

Public data contains ads, reposted content, uniform assets, garbled text, irrelevant content, and anomalous interactions. These cases need to be handled separately.

First filter relevant content. Keywords and dictionaries are responsible for finding as much as possible, while semantic models supplement abbreviations, acronyms, and similar expressions.

Then handle duplicate content. Exact copies, slight rewrites, same image with different text, and republished versions of the same video can all be placed in the same duplicate group. The original records are still retained, and deduplication by post, asset, or account is decided later during statistics.

Finally identify anomalies. Publication time, content similarity, account history, posting rhythm, and interaction networks can all serve as judgment signals.

Risk scores are only responsible for ranking suspicious content first. Unified campaign assets, media matrices, and partner accounts will also be highly similar; you cannot simply delete content just because it has a high score.

"This serum feels quite moisturizing right after application, but gets a bit sticky after a while; sensitive skin may want to patch-test first; at the same price point I'd also like to look at another product."

This sentence contains the current product, candidate products, immediate skin feel, subsequent skin feel, suitable audience, risk warning, price range, and comparison intent. Assigning only a single "negative" or "neutral" label makes it almost unusable downstream.

High-frequency, rule-stable tasks go to rules and small models. Multilingual, multi-object, complex anaphora, and long-text tasks go to large models. Low-confidence results enter manual review.

Large models need to return fields and evidence in a fixed JSON format. Field names, allowed values, and null-value rules must be written clearly in advance so that outputs can be directly accepted and stored.

The team also needs to prepare a set of manually confirmed test samples, including normal data and difficult cases prone to errors. Every time rules or models are changed, rerun the same set of samples.

Finally, when storing data, save the raw data, processing results, quality status, and version information separately. This allows reprocessing, comparison of old and new results, and tracing at which step a piece of data changed.

Finally, take away a general solution

One, define data specifications. Clearly state the purpose, fields, relationships, evidence, and storage conditions.

Two, retain the original content. Keep the original text, original files, links, platform IDs, and collection times.

Three, map out data relationships. Connect users, original posts, comments, replies, and attachments.

Four, complete multimodal normalization. Process text, images, audio, and video separately, then organize them into a unified format.

Five, handle conflicts and quality cleaning well. Handle recognition conflicts, duplicates, missing values, garbled text, and anomalous data.

Six, generate structured data. Extract objects, attributes, relationships, events, opinions, and evidence.

Seven, perform acceptance checks. Use manually confirmed samples to check accuracy and high-risk errors.

Eight, store data according to rules and versions. Save processing rules, model versions, correction reasons, and processing times.

A complete delivery should also include: a data dictionary, relationship documentation, processing workflow, quality report, difficult-case set, review queue, and version records.

What FDE needs to do at this stage is connect business rules, data processing, model capabilities, and manual review into a process that can run long term.

Once the data is cleaned, the next article will address database querying: with Elasticsearch having many fields and deep hierarchies, how AI can reliably retrieve the correct data and write it into a trustworthy report.

Source: @miles_mazy

Miles works on AI algorithm R&D, enterprise AI training, and FDE implementation delivery. Original article "FDE Practical Case | Data Cleaning (How to Clean Massive Multimodal Data into a Trustworthy Dataset)"

www.smartcity.team

Disclaimer: Please indicate the source as "Smart City Industry Analysis (www.smartcity.team)" and the relevant author when reposting. Except for original content, some information on this site comes from the Internet. If the relevant content involves infringement of rights, please contact the contact information at the bottom to delete it.

[Digital Technology Special Research] Community

In-depth peer exchange, focused on smart city and digital technology industry research and material accumulation.

Continuously updating the latest industry white papers, industry research, policy developments, and implementation solutions to help industry practitioners efficiently accumulate knowledge and share experience.

Massive premium content is continuously iterated, providing frontier perspectives, connecting industry talent, carrying out case co-creation and resource exchange, and jointly building a high-quality industry exchange atmosphere.

Ruipu Enterprise Introduction GPS Methodology: Why is your company introduction PPT full of content, yet clients won't listen?

Understand in one article: What are databases, data lakes, data warehouses, lakehouse, and intelligent lakehouse?

[Architect Ramblings] When we talk about transforming business architecture, what are we talking about? — Looking at architecture from the perspectives of business, management, capability, and technology respectively