"
]
},
{
@@ -91,11 +91,11 @@
"id": "dc278740-1511-4423-bf2b-92a5109a47bf",
"metadata": {},
"source": [
- "
Here, we import the required libraries, set environment variables and environment paths (if required).
Here, we import the required libraries, set environment variables and environment paths (if required).
"
]
},
{
@@ -131,10 +131,10 @@
"id": "1763fd95-5000-4a2d-8b86-7be261e20847",
"metadata": {},
"source": [
- "
We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.
We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.
"
]
},
{
@@ -155,7 +155,7 @@
"id": "0ce81699-6345-4dbf-a3a8-b9402c7b6a98",
"metadata": {},
"source": [
- "
Begin running steps with Shift + Enter keys.
Begin running steps with Shift + Enter keys.
"
]
},
{
@@ -163,10 +163,10 @@
"id": "8dca7332-48f5-4702-8b56-d8a03fbb87a2",
"metadata": {},
"source": [
- "
We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.
We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.
"
]
},
{
@@ -185,9 +185,9 @@
"id": "d556a860-e27f-432d-a4d1-f4bd903e34fd",
"metadata": {},
"source": [
- "
"
]
},
{
@@ -206,9 +206,9 @@
"id": "6e3b179f-4649-483a-a470-db064266c3b6",
"metadata": {},
"source": [
- "
The following section defines the type of Gemini model used. Here we use gemini-1.5-pro-latest
The following section defines the type of Gemini model used. Here we use gemini-1.5-pro-latest
"
]
},
{
@@ -221,7 +221,7 @@
"from google.generativeai.types import HarmCategory, HarmBlockThreshold\n",
"\n",
"model = genai.GenerativeModel(\n",
- " model_name = \"models/gemini-1.5-pro-latest\"\n",
+ " model_name = \"models/gemini-2.5-flash\"\n",
")"
]
},
@@ -230,9 +230,9 @@
"id": "4332348c-f45e-4f03-9184-81746d39f566",
"metadata": {},
"source": [
- "
We'll analyze the sentiments of a sample of customer complaints data.
We'll analyze the sentiments of a sample of customer complaints data.
"
]
},
{
@@ -324,7 +324,7 @@
"id": "9162a5c4-a3ee-44c0-be09-5bfaa9b5d827",
"metadata": {},
"source": [
- "
Now the results can be saved back to Vantage.
Now the results can be saved back to Vantage.
"
]
},
{
@@ -358,10 +358,10 @@
"id": "c29ca4fd-3282-4eac-a402-0c72542a5a30",
"metadata": {},
"source": [
- "
A graph illustrating the relationship between consumer sentiments (positive, negative, neutral) prediction and the number of occurrences. This visual representation helps identify trends, patterns, and areas for improvement, enabling data-driven decision making.
A graph illustrating the relationship between consumer sentiments (positive, negative, neutral) prediction and the number of occurrences. This visual representation helps identify trends, patterns, and areas for improvement, enabling data-driven decision making.
"
]
},
{
@@ -385,7 +385,7 @@
" labels={'Count': 'Number of Occurrences', 'Sentiment': 'Sentiment'})\n",
"\n",
"# Show the plot\n",
- "fig.show()"
+ "fig.show(renderer=\"notebook\")"
]
},
{
@@ -393,10 +393,10 @@
"id": "1211a149-ec81-4cd3-8423-963952da4d01",
"metadata": {},
"source": [
- "
Unlock the power of customer feedback with our intuitive word cloud visualization, which provides a comprehensive snapshot of negative consumer complaints sentiment . This innovative tool highlights the most frequently occurring words and pain points in customer feedback, empowering businesses to:
By leveraging this word cloud, businesses can proactively address customer concerns, refine their products and services, and ultimately drive growth through a deeper understanding of their customers' needs and preferences.
Unlock the power of customer feedback with our intuitive word cloud visualization, which provides a comprehensive snapshot of negative consumer complaints sentiment . This innovative tool highlights the most frequently occurring words and pain points in customer feedback, empowering businesses to:
By leveraging this word cloud, businesses can proactively address customer concerns, refine their products and services, and ultimately drive growth through a deeper understanding of their customers' needs and preferences.
"
]
},
{
@@ -409,7 +409,7 @@
"def display_helper(msg):\n",
" return display(Markdown(\n",
" f\"\"\"
\n",
- "
Note: \n",
+ "
Note: \n",
" {msg}
\"\"\"))"
]
},
@@ -444,10 +444,10 @@
"id": "a3c9017a-4470-4184-b7c3-334a2dd7ef4d",
"metadata": {},
"source": [
- "
\n",
- "
4.3 Word Cloud for Neutral Consumer Sentiment Prediction
\n",
+ "
\n",
+ "
4.3 Word Cloud for Neutral Consumer Sentiment Prediction
\n",
"\n",
- "
Tap into the insights of customer feedback with our intuitive word cloud visualization, which offers a detailed overview of neutral consumer complaints sentiment
"
+ "
Tap into the insights of customer feedback with our intuitive word cloud visualization, which offers a detailed overview of neutral consumer complaints sentiment
"
]
},
{
@@ -481,10 +481,10 @@
"id": "d63d3d37-cf05-4497-9d74-a612762fe3e7",
"metadata": {},
"source": [
- "
\n",
- "
4.4 Word Cloud for Positive Consumer Sentiment Prediction
\n",
+ "
\n",
+ "
4.4 Word Cloud for Positive Consumer Sentiment Prediction
\n",
"\n",
- "
Explore customer feedback insights with our intuitive word cloud visualization, providing a detailed overview of consumer sentiment.
"
+ "
Explore customer feedback insights with our intuitive word cloud visualization, providing a detailed overview of consumer sentiment.
"
]
},
{
@@ -519,12 +519,12 @@
"id": "3f81682d-268c-498a-b7d6-fa3d57698fd8",
"metadata": {},
"source": [
- "
\n",
- "
4.5 Negative Sentiment per Product Over Years
\n",
+ "
\n",
+ "
4.5 Negative Sentiment per Product Over Years
\n",
"\n",
- "
This graph tracks the negative sentiment associated with different products over time, offering valuable insights into evolving customer perceptions and pain points.
\n",
+ "
This graph tracks the negative sentiment associated with different products over time, offering valuable insights into evolving customer perceptions and pain points.
\n",
"\n",
- "
We will use Vantage in-db function OrdinalEncodingFit which will identifies distinct categorical values from the input data or a user-defined list and generates the distinct categorical values along with the ordinal value for each category.We will use Vantage in-db function OrdinalEncodingFit which will identifies distinct categorical values from the input data or a user-defined list and generates the distinct categorical values along with the ordinal value for each category.\n",
- "4.6 Neutral Sentiment per Product Over Years
\n",
+ "4.6 Neutral Sentiment per Product Over Years
\n",
"\n",
- "This graph tracks the neutral sentiment associated with different products over time, offering valuable insights into evolving customer perceptions and pain points.
"
+ "This graph tracks the neutral sentiment associated with different products over time, offering valuable insights into evolving customer perceptions and pain points.
"
]
},
{
@@ -625,7 +625,7 @@
" fig = px.line(pd_df_sorted, x='year', y='sum_Sentiment', color='product', markers=True, title='Neutral Sentiment per Product Over Years')\n",
" fig.update_layout(xaxis_title='Year', yaxis_title='Count', legend_title='Product', width=1000, height=600)\n",
"\n",
- " fig.show()\n",
+ " fig.show(renderer=\"notebook\")\n",
"else:\n",
" display_helper(\"\"\"To cover all possible scenarios, we included positive, negative, and neutral categories in our analysis. \n",
" However, given that this dataset consists of complaints, it's expected that the model would rarely, if ever, encounter neutral responses.\"\"\")"
@@ -636,10 +636,10 @@
"id": "389e811a-737d-4953-965b-56b591409aad",
"metadata": {},
"source": [
- "4.7 Positive Sentiment per Product Over Years
\n",
+ "4.7 Positive Sentiment per Product Over Years
\n",
"\n",
- "This graph tracks the positive sentiment associated with different products over time, offering valuable insights into evolving customer perceptions and pain points.
"
+ "This graph tracks the positive sentiment associated with different products over time, offering valuable insights into evolving customer perceptions and pain points.
"
]
},
{
@@ -661,7 +661,7 @@
" fig = px.line(pd_df_sorted, x='year', y='sum_Sentiment', color='product', markers=True, title='Positive Sentiment per Product Over Years')\n",
" fig.update_layout(xaxis_title='Year', yaxis_title='Count', legend_title='Product', width=1000, height=600)\n",
"\n",
- " fig.show()\n",
+ " fig.show(renderer=\"notebook\")\n",
"else:\n",
" display_helper(\"\"\"To cover all possible scenarios, we included positive, negative, and neutral categories in our analysis. \n",
" However, given that this dataset consists of complaints, it's expected that the model would rarely, if ever, encounter positive responses.\"\"\")"
@@ -672,8 +672,8 @@
"id": "a44bc090-a387-41b6-90a5-638a16dc3d4f",
"metadata": {},
"source": [
- "5. Cleanup "
+ "5. Cleanup "
]
},
{
@@ -681,8 +681,8 @@
"id": "3b6ba28d-07d2-4322-9f59-e81855e1389c",
"metadata": {},
"source": [
- " Databases and Tables
\n",
- "The following code will clean up tables and databases created above.
"
+ " Databases and Tables
\n",
+ "The following code will clean up tables and databases created above.
"
]
},
{
@@ -710,11 +710,11 @@
"id": "ebbd137a-7bfc-49fa-a18e-ca34ba68919d",
"metadata": {},
"source": [
- "Dataset: \n",
+ "Dataset: \n",
"The dataset is sourced from Consumer Financial Protection Bureau
"
+ "The dataset is sourced from Consumer Financial Protection Bureau
"
]
},
{
@@ -722,11 +722,11 @@
"id": "19bdc0df-dc13-4a13-bf73-c7039a54c3ba",
"metadata": {},
"source": [
- "\n",
+ ""
@@ -749,7 +749,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.10"
+ "version": "3.11.14"
}
},
"nbformat": 4,
diff --git a/UseCases/Complaints_Analysis_GenAI_Gemini/Speech_Recognition.ipynb b/UseCases/Complaints_Analysis_GenAI_Gemini/Speech_Recognition.ipynb
index 6ce4ff15..c918eee5 100644
--- a/UseCases/Complaints_Analysis_GenAI_Gemini/Speech_Recognition.ipynb
+++ b/UseCases/Complaints_Analysis_GenAI_Gemini/Speech_Recognition.ipynb
@@ -19,13 +19,13 @@
"id": "f009bba8-cfaa-4910-a5fe-c559c7d92ae8",
"metadata": {},
"source": [
- "Introduction:
\n",
+ "Introduction:
\n",
"\n",
- "Speech Recognition using Teradata Vantage on Google and Google Gemini LLM model is a cutting-edge technology that leverages the power of cloud computing and advanced algorithms to accurately recognize and transcribe spoken words. This innovative solution integrates the capabilities of Teradata Vantage, Google Cloud, and Gemini to provide a robust and efficient speech recognition system.
\n",
+ "Speech Recognition using Teradata Vantage on Google and Google Gemini LLM model is a cutting-edge technology that leverages the power of cloud computing and advanced algorithms to accurately recognize and transcribe spoken words. This innovative solution integrates the capabilities of Teradata Vantage, Google Cloud, and Gemini to provide a robust and efficient speech recognition system.
\n",
"\n",
- "Key Features:
\n",
+ "Key Features:
\n",
"\n",
- "\n",
+ "\n",
" Cloud-Based Architecture : Teradata Vantage provides a scalable and secure cloud infrastructure, allowing for seamless integration with Google Cloud and Gemini. This architecture ensures high availability, reliability, and flexibility for speech recognition applications.Advanced Algorithms : The system employs sophisticated machine learning algorithms, leveraging Google's expertise in AI and machine learning. These algorithms are trained on vast amounts of data, enabling the system to recognize speech patterns, nuances, and variations with high accuracy.Real-Time Transcription : The speech recognition system can transcribe spoken words in real-time, making it ideal for applications such as virtual assistants, customer service chatbots, and live captioning.Cost-Effective : The cloud-based model provides a cost-effective solution, eliminating the need for on-premise infrastructure and reducing operational costs. \n",
"\n",
- "By combining the strengths of Teradata Vantage , on Google and Gemini , this speech recognition system offers a powerful and versatile solution for a wide range of applications, from customer service to healthcare and beyond.
\n",
+ "By combining the strengths of Teradata Vantage , on Google and Gemini , this speech recognition system offers a powerful and versatile solution for a wide range of applications, from customer service to healthcare and beyond.
\n",
"\n",
- "Steps in the analysis:
\n",
- "\n",
+ "Steps in the analysis:
\n",
+ "\n",
" Configuring the environment \n",
" Connect to Vantage \n",
" Setup API key for Google Gemini \n",
@@ -56,10 +56,10 @@
"id": "4292dcda-26f7-4a06-a471-19cd09b32d00",
"metadata": {},
"source": [
- "1. Configuring the environment \n",
+ "1. Configuring the environment \n",
"\n",
- "1.1 Install the required libraries
"
+ "1.1 Install the required libraries
"
]
},
{
@@ -81,7 +81,7 @@
"metadata": {},
"source": [
"\n",
- "
Note: Please restart the kernel after executing these two lines. The simplest way to restart the Kernel is by typing zero zero: 0 0
"
+ "
Note: Please restart the kernel after executing these two lines. The simplest way to restart the Kernel is by typing zero zero: 0 0
"
]
},
{
@@ -89,11 +89,11 @@
"id": "c233ee65-c69c-4ea4-8964-d4b53acdb5cd",
"metadata": {},
"source": [
- "
\n",
+ "
\n",
"\n",
- "
1.2 Import the required libraries
\n",
+ "
1.2 Import the required libraries
\n",
"\n",
- "
Here, we import the required libraries, set environment variables and environment paths (if required).
"
+ "
Here, we import the required libraries, set environment variables and environment paths (if required).
"
]
},
{
@@ -116,10 +116,10 @@
"id": "3fdd09db-eca3-4970-9bf5-f8d22da9d115",
"metadata": {},
"source": [
- "
\n",
- "
2. Connect to Vantage \n",
- "
2.1 Connect to Vantage
\n",
- "
We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.
"
+ "
\n",
+ "
2. Connect to Vantage \n",
+ "
2.1 Connect to Vantage
\n",
+ "
We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.
"
]
},
{
@@ -139,7 +139,7 @@
"id": "b5e8fae0-5b96-41df-88a7-0f8b6a36362c",
"metadata": {},
"source": [
- "
Begin running steps with Shift + Enter keys.
"
+ "
Begin running steps with Shift + Enter keys.
"
]
},
{
@@ -147,10 +147,10 @@
"id": "b562b3e9-d58c-4220-b23f-3d22998cf5e0",
"metadata": {},
"source": [
- "
\n",
+ "
\n",
"\n",
- "
2.2 Getting Data for This Demo
\n",
- "
We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.
"
+ "
2.2 Getting Data for This Demo
\n",
+ "
We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.
"
]
},
{
@@ -169,9 +169,9 @@
"id": "5a57f400-6d82-4f0c-b9d5-d8048bf48a31",
"metadata": {},
"source": [
- "
\n",
- "
3. Setup API key for Google Gemini \n",
- "
Please enter the Google API Key, if you don't have one, please get it from here
"
+ "
\n",
+ "
3. Setup API key for Google Gemini \n",
+ "
Please enter the Google API Key, if you don't have one, please get it from here
"
]
},
{
@@ -190,9 +190,9 @@
"id": "3d1b1d2e-3d79-4e79-851d-a1d3a342777c",
"metadata": {},
"source": [
- "
\n",
- "
4. Define the Gemini model and Prompt \n",
- "
The following section defines the type of Gemini model used. Here we use gemini-1.5-pro-latest
"
+ "
\n",
+ "
4. Define the Gemini model and Prompt \n",
+ "
The following section defines the type of Gemini model used. Here we use gemini-1.5-pro-latest
"
]
},
{
@@ -205,7 +205,7 @@
"from google.generativeai.types import HarmCategory, HarmBlockThreshold\n",
"\n",
"model = genai.GenerativeModel(\n",
- " model_name = \"models/gemini-1.5-pro-latest\", \n",
+ " model_name = \"models/gemini-2.5-flash\", \n",
" safety_settings = {\n",
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
" HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
@@ -270,7 +270,7 @@
"id": "afed5b12-238b-4342-b2e5-09f339e61708",
"metadata": {},
"source": [
- "
The following cell uploads the audio file to Google Gemini.
"
+ "
The following cell uploads the audio file to Google Gemini.
"
]
},
{
@@ -294,11 +294,11 @@
"id": "a0722f06-3825-49f2-8219-e3e4f3a3da4d",
"metadata": {},
"source": [
- "
\n",
- "
5. Speech Recognition \n",
- "
The following section calls Google Gemini with the above prompt and analyzes the audio file that we uploaded.
\n",
+ "
\n",
+ "
5. Speech Recognition \n",
+ "
The following section calls Google Gemini with the above prompt and analyzes the audio file that we uploaded.
\n",
"\n",
- "
Speech Recognition using Large Language Models (LLMs) is a revolutionary technology that leverages the power of artificial intelligence to accurately recognize and transcribe spoken words. By harnessing the capabilities of LLMs, this system can learn from vast amounts of data, adapt to various speech patterns, and improve its accuracy over time. With its ability to recognize speech in real-time, this technology has numerous applications, including virtual assistants, customer service chatbots, and live captioning, making it an essential tool for enhancing user experiences and improving communication.
"
+ "
Speech Recognition using Large Language Models (LLMs) is a revolutionary technology that leverages the power of artificial intelligence to accurately recognize and transcribe spoken words. By harnessing the capabilities of LLMs, this system can learn from vast amounts of data, adapt to various speech patterns, and improve its accuracy over time. With its ability to recognize speech in real-time, this technology has numerous applications, including virtual assistants, customer service chatbots, and live captioning, making it an essential tool for enhancing user experiences and improving communication.
"
]
},
{
@@ -331,10 +331,10 @@
"id": "128480fd-a514-422f-9cd4-c3230f2f5856",
"metadata": {},
"source": [
- "
\n",
- "
6. Cleanup \n",
+ "
\n",
+ "
6. Cleanup \n",
"\n",
- "
The following code will clean up the audio files uploaded to Google Gemini.
"
+ "
The following code will clean up the audio files uploaded to Google Gemini.
"
]
},
{
@@ -353,8 +353,8 @@
"id": "7cc8a5e4-f87d-4b34-bd26-00cf808b1e21",
"metadata": {},
"source": [
- "
Databases and Tables
\n",
- "
The following code will clean up tables and databases created above.
"
+ "
Databases and Tables
\n",
+ "
The following code will clean up tables and databases created above.
"
]
},
{
@@ -382,11 +382,11 @@
"id": "50c05788-5048-42fd-8327-ae8e1c52e114",
"metadata": {},
"source": [
- "
\n",
+ ""
@@ -409,7 +409,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.10"
+ "version": "3.11.14"
}
},
"nbformat": 4,
diff --git a/UseCases/Complaints_Analysis_GenAI_Gemini/Topic_Modelling.ipynb b/UseCases/Complaints_Analysis_GenAI_Gemini/Topic_Modelling.ipynb
index 03634bb1..15bdc777 100644
--- a/UseCases/Complaints_Analysis_GenAI_Gemini/Topic_Modelling.ipynb
+++ b/UseCases/Complaints_Analysis_GenAI_Gemini/Topic_Modelling.ipynb
@@ -19,30 +19,30 @@
"id": "a1cca519-4be2-49f4-85f6-c81f26a6c4fe",
"metadata": {},
"source": [
- "Introduction:
\n",
+ "Introduction:
\n",
"\n",
- "In this comprehensive user demo, we will delve into the world of topic modeling using Teradata Vantage and Google Gemini LLM model . This cutting-edge technology empowers businesses to uncover hidden insights from vast amounts of consumer complaints data, enabling them to identify trends, improve customer satisfaction, and enhance their overall brand reputation.
\n",
+ "In this comprehensive user demo, we will delve into the world of topic modeling using Teradata Vantage and Google Gemini LLM model . This cutting-edge technology empowers businesses to uncover hidden insights from vast amounts of consumer complaints data, enabling them to identify trends, improve customer satisfaction, and enhance their overall brand reputation.
\n",
"\n",
- "Key Features:
\n",
+ "Key Features:
\n",
"\n",
- " \n",
+ " \n",
" Scalable Data Ingestion : Seamlessly integrate and process large volumes of consumer complaints data from various sources, including Google Gemini, into Teradata Vantage.Advanced Topic Modelling : Utilize state-of-the-art topic modeling algorithms to identify and categorize underlying themes and sentiments within the complaints data, providing actionable insights.Real-time Analytics : Leverage Teradata Vantage's real-time analytics capabilities to monitor and respond to emerging trends and issues in consumer complaints.Customizable Dashboards : Create tailored dashboards to visualize and track key performance indicators (KPIs) and metrics specific to your business needs.Integration with Google Gemini : Seamlessly integrate with Google Gemini to collect and analyze consumer complaints data from these platforms. \n",
"\t\n",
- "Benefits:
\n",
+ "Benefits:
\n",
"\t\n",
- "\n",
+ "\n",
"Enhanced Customer Insights : Gain a deeper understanding of customer concerns and preferences, enabling data-driven decision-making.Improved Customer Satisfaction : Identify and address recurring issues, leading to increased customer satisfaction and loyalty.Competitive Advantage : Stay ahead of the competition by proactively addressing consumer complaints and improving brand reputation.Cost Savings : Reduce the financial burden of handling and resolving consumer complaints by identifying and addressing root causes.Data-Driven Decision-Making : Make informed business decisions based on actionable insights derived from topic modeling and real-time analytics. \n",
"\n",
- "Steps in the analysis:
\n",
- "\n",
+ "Steps in the analysis:
\n",
+ "\n",
" Configuring the environment \n",
" Connect to Vantage \n",
" Setup API key for Google Gemini \n",
@@ -57,10 +57,10 @@
"id": "6d7d46c0-b2e2-4ffd-80ad-255bfff17086",
"metadata": {},
"source": [
- "1. Configuring the environment \n",
+ "1. Configuring the environment \n",
"\n",
- "1.1 Install the required libraries
"
+ "1.1 Install the required libraries
"
]
},
{
@@ -81,7 +81,7 @@
"metadata": {},
"source": [
"\n",
- "
Note: Please restart the kernel after executing these two lines. The simplest way to restart the Kernel is by typing zero zero: 0 0
"
+ "
Note: Please restart the kernel after executing these two lines. The simplest way to restart the Kernel is by typing zero zero: 0 0
"
]
},
{
@@ -89,11 +89,11 @@
"id": "b41b93c3-f22d-4017-9e93-749f7492a6b3",
"metadata": {},
"source": [
- "
\n",
+ "
\n",
"\n",
- "
1.2 Import the required libraries
\n",
+ "
1.2 Import the required libraries
\n",
"\n",
- "
Here, we import the required libraries, set environment variables and environment paths (if required).
"
+ "
Here, we import the required libraries, set environment variables and environment paths (if required).
"
]
},
{
@@ -126,10 +126,10 @@
"id": "3cc65823-8d76-4ce4-90c0-7005679e8dcc",
"metadata": {},
"source": [
- "
\n",
- "
2. Connect to Vantage \n",
- "
2.1 Connect to Vantage
\n",
- "
We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.
"
+ "
\n",
+ "
2. Connect to Vantage \n",
+ "
2.1 Connect to Vantage
\n",
+ "
We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.
"
]
},
{
@@ -150,7 +150,7 @@
"id": "cdd806c9-dcd4-4b6a-ae42-854b9cc74187",
"metadata": {},
"source": [
- "
Begin running steps with Shift + Enter keys.
"
+ "
Begin running steps with Shift + Enter keys.
"
]
},
{
@@ -158,10 +158,10 @@
"id": "afb8acf9-78d7-4bdd-888d-9585ccce9907",
"metadata": {},
"source": [
- "
\n",
+ "
\n",
"\n",
- "
2.2 Getting Data for This Demo
\n",
- "
We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.
"
+ "
2.2 Getting Data for This Demo
\n",
+ "
We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.
"
]
},
{
@@ -180,9 +180,9 @@
"id": "d9ad01a0-8bc7-45b8-a137-11470fd4f997",
"metadata": {},
"source": [
- "
\n",
- "
3. Setup API key for Google Gemini \n",
- "
Please enter the Google API Key, if you don't have one, please get it from here
"
+ "
\n",
+ "
3. Setup API key for Google Gemini \n",
+ "
Please enter the Google API Key, if you don't have one, please get it from here
"
]
},
{
@@ -201,9 +201,9 @@
"id": "04a3a68e-0217-47b1-8402-9670c45098bf",
"metadata": {},
"source": [
- "
\n",
- "
3.1. Define the Gemini model and Prompt \n",
- "
The following section defines the type of Gemini model used. Here we use gemini-1.5-pro-latest
"
+ "
\n",
+ "
3.1. Define the Gemini model and Prompt \n",
+ "
The following section defines the type of Gemini model used. Here we use gemini-1.5-pro-latest
"
]
},
{
@@ -216,7 +216,7 @@
"from google.generativeai.types import HarmCategory, HarmBlockThreshold\n",
"\n",
"model = genai.GenerativeModel(\n",
- " model_name = \"models/gemini-1.5-pro-latest\"\n",
+ " model_name = \"models/gemini-2.5-flash\"\n",
")"
]
},
@@ -225,8 +225,8 @@
"id": "a9cabc44-d40d-4f27-ab0a-cfc09315e179",
"metadata": {},
"source": [
- "
\n",
- "
4. Exploring the data "
+ "
\n",
+ "
4. Exploring the data "
]
},
{
@@ -244,7 +244,7 @@
"id": "aa2e019c-086c-4574-bda8-27494c64c090",
"metadata": {},
"source": [
- "
Here we subset the data to get only the complaints related to Mortgage. We further analyze the issues of those complaints and pick the top 5 topics.
"
+ "
Here we subset the data to get only the complaints related to Mortgage. We further analyze the issues of those complaints and pick the top 5 topics.
"
]
},
{
@@ -272,9 +272,9 @@
"id": "deeba736-584f-4fff-8439-fe934764722e",
"metadata": {},
"source": [
- "
According to the result above, we can classify the issues into the following topics:
\n",
+ "
According to the result above, we can classify the issues into the following topics:
\n",
"\n",
- "
\n",
+ "\n",
" Mortgage Application : Applying or refinancingPayment Trouble : Issues during paymentMortgage Closing : Finalizing the mortgage5. Topic Modelling \n",
+ "5. Topic Modelling \n",
"\n",
- "Topic modeling using Large Language Models (LLMs) revolutionizes the way we understand and categorize vast collections of text data. LLMs excel in understanding the semantics and context of words, enabling sophisticated topic modeling techniques.
\n",
+ "Topic modeling using Large Language Models (LLMs) revolutionizes the way we understand and categorize vast collections of text data. LLMs excel in understanding the semantics and context of words, enabling sophisticated topic modeling techniques.
\n",
"\n",
- "Traditionally, topic modeling algorithms like Latent Dirichlet Allocation (LDA) rely on statistical patterns within documents to identify topics. However, LLMs offer a more nuanced approach. By leveraging their deep understanding of language, LLMs can extract complex themes and topics from unstructured text data with higher accuracy and flexibility.
\n",
+ "Traditionally, topic modeling algorithms like Latent Dirichlet Allocation (LDA) rely on statistical patterns within documents to identify topics. However, LLMs offer a more nuanced approach. By leveraging their deep understanding of language, LLMs can extract complex themes and topics from unstructured text data with higher accuracy and flexibility.
\n",
"\n",
- "LLMs can generate coherent topics without needing predefined categories, making them ideal for exploratory analysis of diverse datasets. Moreover, their ability to capture subtle nuances in language allows for more precise topic identification, even in noisy or ambiguous texts.
\n",
+ "LLMs can generate coherent topics without needing predefined categories, making them ideal for exploratory analysis of diverse datasets. Moreover, their ability to capture subtle nuances in language allows for more precise topic identification, even in noisy or ambiguous texts.
\n",
"\n",
- "Reasoning with a Chain of Thought : Imagine you're trying to solve a problem. With a large language model, you start with an initial idea or question. Then, you use the model's capabilities to explore related concepts, gradually connecting them together. Each step builds upon the previous one, leading you closer to understanding or solving the problem. It's like putting together puzzle pieces, one by one, until you see the whole picture.
"
+ "Reasoning with a Chain of Thought : Imagine you're trying to solve a problem. With a large language model, you start with an initial idea or question. Then, you use the model's capabilities to explore related concepts, gradually connecting them together. Each step builds upon the previous one, leading you closer to understanding or solving the problem. It's like putting together puzzle pieces, one by one, until you see the whole picture.
"
]
},
{
@@ -383,7 +383,7 @@
"id": "00cb0cb4-572d-49c4-9068-c4dcfc326c7e",
"metadata": {},
"source": [
- "Now the results can be saved back to Vantage.
"
+ "Now the results can be saved back to Vantage.
"
]
},
{
@@ -401,10 +401,10 @@
"id": "ea01a05e-2116-4c7c-94db-ac369a14231f",
"metadata": {},
"source": [
- "5.1 Number of Complaints by Predicted Topic
\n",
+ "5.1 Number of Complaints by Predicted Topic
\n",
"\n",
- "A graph illustrating the Number of Complaints by Predicted Topic reveals that the majority of complaints are centered around Mortgage Application, while the fewest are related to Mortgage Closing.
"
+ "A graph illustrating the Number of Complaints by Predicted Topic reveals that the majority of complaints are centered around Mortgage Application, while the fewest are related to Mortgage Closing.
"
]
},
{
@@ -425,7 +425,7 @@
"# Add hover information\n",
"fig.update_traces(hovertemplate='Issue: %{x}6. Cleanup "
+ "6. Cleanup "
]
},
{
@@ -442,8 +442,8 @@
"id": "07a477ec-ea8e-4394-afcb-3fce4f3c0d19",
"metadata": {},
"source": [
- " Databases and Tables
\n",
- "The following code will clean up tables and databases created above.
"
+ " Databases and Tables
\n",
+ "The following code will clean up tables and databases created above.
"
]
},
{
@@ -471,11 +471,11 @@
"id": "b6cbe888-1ced-464e-b5e4-05d6af91dd5e",
"metadata": {},
"source": [
- "Dataset: \n",
+ "Dataset: \n",
"The dataset is sourced from Consumer Financial Protection Bureau
"
+ "The dataset is sourced from Consumer Financial Protection Bureau
"
]
},
{
@@ -483,11 +483,11 @@
"id": "6c5372e5-c7c9-4e9e-908a-6fba19b4e022",
"metadata": {},
"source": [
- "\n",
+ ""
@@ -510,7 +510,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.10"
+ "version": "3.11.14"
}
},
"nbformat": 4,
diff --git a/UseCases/Complaints_Analysis_GenAI_Gemini/data/embeddings.parquet.gzip b/UseCases/Complaints_Analysis_GenAI_Gemini/data/embeddings.parquet.gzip
index ce8be44e..5af84ca2 100644
Binary files a/UseCases/Complaints_Analysis_GenAI_Gemini/data/embeddings.parquet.gzip and b/UseCases/Complaints_Analysis_GenAI_Gemini/data/embeddings.parquet.gzip differ