Machine Learning and Deep Learning Solutions (NLP) for Automating Market Research Processes
Market research is a domain teeming with diverse data types, including tabular, audio, and particularly text. This wealth of data can be utilised not only to enhance internal processes through automation but also to equip the end client with valuable new functionalities and use cases.
The objective of this use case was to automate the labelling of open-ended questions from online market research surveys. Open-ended questions, which receive text responses, require further post-processing to extract useful information. I automated this process for one of the world's largest market research firms using a series of interconnected techniques:
Utilising various forms of word embeddings, leveraging both Neural Network models and general techniques.
Implementing clustering to group similar responses.
Applying summarization and content extraction methods to distil and highlight key insights.
About expert
A Machine Learning Engineer at Kantar.
Alin-Gabriel applied their knowledge of Natural Language Processing, Machine learning, Artificial Intelligence and Deep Learning in Market Research on the following markets: Romania.
Alin-Gabriel built products like Content Extraction System and Survey Analysis Tool using the variety of tools.
Other cases by Alin-Gabriel
Data Flow Enhancement and Efficient Processing through Large Language Models (LLMs)
In the corporate sector, text data flows swiftly, emanating from diverse sources in various formats, destined for multiple endpoints and actions. This continuous stream of data necessitates management through structuring, making it easy to interpret and analyse, and ultimately directing it to the appropriate user or applying the insights obtained. I utilised several large language models (LLMs) streamline and enhance this process:
A question-answering LLM, enabling users to inquire about the content of each document within the flow.
A summarization LLM, offering a concise overview of each document.
An NLP model for sentiment analysis, determining the sentiment of each document.
An NLP model that calculates an absolute similarity score among all documents.
Clustering, which groups similar documents together and simultaneously identifies outlier documents.
ML and DL Solutions for Banking Anomaly Detection
In the dynamic realm of banking, ensuring effective anomaly detection is paramount. Utilising my expertise in state-of-the-art ML algorithms and advanced Deep Learning techniques, I tackled this imperative at Deutsche Bank, a globally recognized leader in financial services. The solution involved several critical stages:
Identifying relevant data from various sources.
Researching and testing optimal Machine Learning algorithms for tabular data analysis.
Exploring and validating appropriate Deep Learning models for extracting insights from textual data (Natural Language Processing).
Integrating the resulting models into the rigorous banking environment, complete with all necessary software engineering components: Python, Object-Oriented Programming, Algorithm Design and Complexity, Deployment, and more.
Similar cases
Setting up efficient customer support
Organized pre-launch product UX testing, regular users research and new features backlog prioritization
Organized the company's participation in high-profile contests, resulting in Nimb becoming a winner of Smart City Expo worldwide Congress’s call for solution in the “Safe Cities” category and a finalist of Xprize Women’s Safety competition
Set up efficient client support
Enhancing Analytical Infrastructure and Reducing Fraudulent Activities
Led the shift of analytical infrastructure, enhancing data resources and decreasing update errors by 90%. Increased identifiable traffic sources from 60% to 95%. Played a crucial role in reducing fraudulent responses from 28% to 1-4% by strengthening moderation and creating a fraud analytics direction. Contributed to a project reducing the cost of applications from groups by 70% and reducing vacancies in central Moscow from 20% to 6%.
