Tutorial
Implementing the Apriori algorithm in R on watsonx.ai
Finding and ranking item sets, and visualizing association rules to show relationshipsThe Apriori algorithm is a popular data mining algorithm to identify frequent item sets, which are a group of items that occur together in a data set. Item sets are important for identifying similarities across events, discrete classes, or categories. For examples, shoppers at a hardware store might buy both a hammer and nails or both paint and paintbrushes.
In this tutorial, you'll learn the fundamentals of implementing an Apriori algorithm in R using a Jupyter Notebook on IBM watsonx.ai platform. We will use a Groceries Data set for classifying item sets from customer purchase histories.
More about the Apriori algorithm
The Apriori algorithm is an unsupervised machine learning algorithm that excels at association rule mining, which is a technique that identifies complex interrelations between different variables of large data sets. It's unsupervised because it doesn't require any labels in order to process the data and create associations from that data. The technique identifies frequent patterns, connections, and dependencies among candidate item sets. The Apriori algorithm runs in iterations over the transaction data to identify k item sets, meaning k items that frequently occur together.
The Apriori algorithm is based on the Apriori property which states that if an item set appears frequently in a data set then all of its subsets must also be frequent. Conversely, if an item set is identified as infrequent, all its supersets will also be considered infrequent. This means that the most frequent items will be weighted more heavily for forming the association rule learning whereas less frequent items will be weighted less heavily.
The Apriori algorithm is widely used for data analysis, especially transactional data analysis. Let's look at a few use cases:
When running a market basket analysis, where retail stores examine customer purchase history to identify items frequently bought together and then optimize shelf space by placing those items closer together.
Recommender systems can be improved by studying product-based relationships based on user preferences and purchase patterns.
To identify fraudulent patterns in financial transactions, for example, where particular combinations of items might indicate fraudelent activity.
In web usage mining, we can analyze clickstream data and user behavior and then apply that information to improve recommender systems.
In healthcare, the Apriori algorithm can help dentify strong association rules between symptoms and diseases, enhancing the efficiency of diagnosis and devising targeted treatment plans.
To help analyze data patterns in unsupervised learning-based artificial intelligence applications like clustering algorithms.
One of the most significant advantages of using the Apriori algorithm is its simplicity and adaptability. Apriori algorithms are not as efficient when handling large data sets because the multi-iteration process of item set candidate generation can become computationally expensive and memory-intensive. A combination of Apriori with other techniques is often used to mitigate these performance issues.
The Apriori algorithm function is well-established and integrated into many software libraries of popular programming languages like Python, Java, and R. Its reliable implementation efficiently generates quality association rules and frequent item sets.
Prerequisites
You need an IBM Cloud account to create a watsonx.ai project.
Steps
We'll use the Apriori algorithm to analyze the transactions in this data set and identify relationships between items bought together. The steps include importing the libraries, loading the data set, exploring it, cleaning out the unnecessary values, preparing the data set into the required format, and then applying the Apriori algorithm. We'll also visualize the efficacy of the association rules evaluated by the model.
Step 1. Set up your environment
While you can choose from several tools, this tutorial walks you through how to set up an IBM account to use a Jupyter Notebook. Jupyter Notebooks are widely used within data science to combine code, text, images, and data visualizations to formulate a well-formed analysis.
Log in to watsonx.ai using your IBM Cloud account.
Create a watsonx.ai project.
Create a Jupyter Notebook using the R Kernel.
This step will open a Notebook environment where you can load your data set and copy the code from this tutorial to perform the association rule mining process.
Step 2. Load libraries
The primary package for performing apriori analysis in the R programming language is the arules package. We can also use the arulesViz package to create visualizations of our rules. We'll use the plyr and dplyr libraries for data manipulation and finally the ggplot library to help us graph some exploratory data analysis.
install.packages('arules')
install.packages('arulesViz')
install.packages('plyr')
install.packages('dplyr')
install.packages('ggplot2')
library(arules)
library(arulesViz)
library(plyr)
library(dplyr)
library(ggplot2)
Step 3. Getting the data
I used the Grocery data set downloaded from Kaggle, which contains 38765 rows of data from customers’ grocery store purchase orders. To access the data set, visit the Groceries Data set page and download the zip file there. Then, upload the zip to your watsonx notebook.
groceries <- read.csv("Groceries_dataset.csv")
head(groceries)
This gives us the following:
Member_number Date itemDescription
1 1808 21-07-2015 tropical fruit
2 2552 05-01-2015 whole milk
3 2300 19-09-2015 pip fruit
4 1187 12-12-2015 other vegetables
5 3037 01-02-2015 whole milk
6 4941 14-02-2015 rolls/buns
Step 4: Exploratory Data Analysis
As a very first step, we should make sure that our data set does not contain null values.
#checking missing values
colSums(is.na(groceries))
So far, so good!
Now, let's look at the number of transactions per user. In order to develop good association rules, we'll want to have adequate transaction data. We know we have enough records, but do we have enough purchases per customer?
# Aggregating and sorting the data
groceries_agg <- groceries %>%
group_by(Member_number) %>%
dplyr::summarise(., count = n()) %>%
arrange(count)
ggplot(groceries_agg, aes(x = count)) +
geom_histogram(aes(y = after_stat(density)), binwidth = 1, colour = "#333333", fill = "#FFFFFF") +
geom_density(alpha = .2, fill = "#3399FF") +
ggtitle("Distribution of Grocery Purchases by Customer") +
xlab("Number of Purchases") +
ylab("Density") +
theme_minimal()
We can see that we have a long tail of purchases, but that it's likely that the shopping carts that we can infer from our purchase histories will be large enough to generate meaningful rules.
Step 5: Data Preparation
What we need now is a data.frame that represents the items purchased in a single session. Since our data is organized by items and then date and member number, we'll collapse all items that match the date and member number into a single item set. We'll use the ddply function to build out the data structure that we want:
transactions <- plyr::ddply(groceries, c("Member_number", "Date"), function(df) paste(df$itemDescription, collapse = ","))
transactions$V1 <- gsub("\"", "", transactions$V1) # get rid of extra " characters
transactions$cart <- paste(transactions$Date, transactions$Member_number, sep = "-") #combine into a cart
head(transactions)
This gives us a cleaned up data set:
Member_number Date V1 cart
1 1000 15-03-2015 sausage,whole milk,semi-finished bread,yogurt 15-03-2015-1000
2 1000 24-06-2014 whole milk,pastry,salty snack 24-06-2014-1000
3 1000 24-07-2015 canned beer,misc. beverages 24-07-2015-1000
4 1000 25-11-2015 sausage,hygiene articles 25-11-2015-1000
5 1000 27-05-2015 soda,pickled vegetables 27-05-2015-1000
6 1001 02-05-2015 frankfurter,curd 02-05-2015-1001
Now we can write out a csv file:
transactions$Member_number <- NULL
transactions$Date <- NULL
write.csv(transactions, file = "transactions.csv", quote = FALSE, row.names = TRUE)
Step 6: Creating the Association Rules
When we read the transaction data back in from the CSV file it will create a transaction object used by the arules library to generate the association rules.
data <- arules::read.transactions(file = "transactions.csv", rm.duplicates = TRUE, format = "basket", sep = ",", cols = 1)
We can view the frequency of different items using the itemFrequencyPlot method:
itemFrequencyPlot(data, support=0.05)
This gives a sense of what the most frequent items are and what items we should expect to see most commonly in our rules. Note the prevalance of whole milk and other vegetables, we'll be seeing both of these quite a bit when we generate rules. Now that we have our data in the correct format, it's time to take a look at how we generate the rules themselves using th apriori algorithm.
The arules library defines a method called apriori which we'll use to generate our rules. There are a few important parameters that we can set for how apriori generates the association rules:
conf: a numeric value for the minimum confidence of rules/association connection. In association rules parlance these are called "hyperedges"
minlen: an integer value for the minimal number of items per item set (default: 1 item). In our case, we'll be looking for item sets with just 2 items. When you run this code, try changing the value of minlen to 3 and see how it changes the generated rules.
sup: You can indicate a required minimum support threshold when applying the Apriori algorithm. This means that any item or item set with "support" less than the specified minimum support will be considered infrequent. You can set a minimum support threshold using the argument min_support, which will filter out the less frequently purchased items. The smaller the minimum support threshold value, the more confidently the algorithm will identify relations. For this tutorial, we'll set the sup value at "0.002."
rules <- arules::apriori(data, parameter = list(minlen = 2, sup = 0.002, conf = 0.02, target = "rules"))
This returns the following:
Apriori
Parameter specification:
confidence minval smax arem aval originalSupport maxtime support minlen maxlen target ext
0.02 0.1 1 none FALSE TRUE 5 0.002 2 10 rules TRUE
Algorithmic control:
filter tree heap memopt load sort verbose
0.1 TRUE TRUE FALSE TRUE 2 TRUE
Absolute minimum support count: 29
set item appearances ...[0 item(s)] done [0.00s].
set transactions ...[15132 item(s), 14964 transaction(s)] done [0.01s].
sorting and recoding items ... [126 item(s)] done [0.00s].
creating transaction tree ... done [0.00s].
checking subsets of size 1 2 3 done [0.00s].
writing ... [382 rule(s)] done [0.00s].
creating S4 object ... done [0.00s].
The apriori method returns the rules object which, as we can see above, contains 382 rules generated from our data set according to the parameters passed to the method.
We can look at the rules to see the sets of items contained in the generated rules and information about the strength of those associations and rules:
arules::inspect(rules[1:20])
This returns the following:
lhs rhs support confidence coverage lift count
[1] {candy} => {whole milk} 0.002138466 0.14883721 0.01436782 0.9425307 32
[2] {meat} => {other vegetables} 0.002138466 0.12698413 0.01684042 1.0400605 32
[3] {meat} => {whole milk} 0.002205293 0.13095238 0.01684042 0.8292727 33
[4] {ham} => {whole milk} 0.002739909 0.16015625 0.01710773 1.0142100 41
[5] {frozen meals} => {other vegetables} 0.002138466 0.12749004 0.01677359 1.0442041 32
[6] {sugar} => {whole milk} 0.002472601 0.13962264 0.01770917 0.8841783 37
[7] {long life bakery product} => {whole milk} 0.002405774 0.13432836 0.01790965 0.8506515 36
[8] {waffles} => {whole milk} 0.002606255 0.14079422 0.01851109 0.8915974 39
[9] {salty snack} => {other vegetables} 0.002205293 0.11743772 0.01877840 0.9618709 33
[10] {onions} => {whole milk} 0.002940390 0.14521452 0.02024860 0.9195895 44
[11] {UHT-milk} => {other vegetables} 0.002138466 0.10000000 0.02138466 0.8190476 32
[12] {UHT-milk} => {whole milk} 0.002539428 0.11875000 0.02138466 0.7519996 38
[13] {berries} => {other vegetables} 0.002673082 0.12269939 0.02178562 1.0049664 40
[14] {other vegetables} => {berries} 0.002673082 0.02189381 0.12209302 1.0049664 40
[15] {berries} => {whole milk} 0.002272120 0.10429448 0.02178562 0.6604581 34
[16] {hamburger meat} => {other vegetables} 0.002205293 0.10091743 0.02185245 0.8265618 33
[17] {hamburger meat} => {whole milk} 0.003074044 0.14067278 0.02185245 0.8908284 46
[18] {napkins} => {other vegetables} 0.002138466 0.09667674 0.02211975 0.7918285 32
[19] {napkins} => {whole milk} 0.002405774 0.10876133 0.02211975 0.6887450 36
[20] {dessert} => {rolls/buns} 0.002004812 0.08498584 0.02358995 0.7726173 30
The lhs is what's called the antecedant and it's the first item in the item set. The rhs is the second item (or items) are called the precendent and they're what the antecedent implies. Another way of say that is that in the first rule we see that if someone buys candy then they're also likely to have bought milk as well. However, when we look at the lift parameter, we see that they're probably not buying the milk because of the candy but instead because of something else in their cart. Let's look at what these metrics mean:
Support: This is a measure of how popular an item set is, as measured by the proportion of transactions in which an item set appears.
Confidence: This shows how likely item Y is purchased when item X is purchased, expressed as {X -> Y}. This is measured by the proportion of transactions with item X in which item Y also appears.
Lift: This is a measure of the effect that one item has on the other items in the item set. Let's say we have two items, A and B. The lift of the antecedent 'A' to the consequent 'B' is calculated as Support(A ∪ B) / (Support(A) * Support(B)). This could be described as "the support of A given B divided by the support for A times the support for B". We interpret the values like this:
- If lift = 1, A and B are completely independent. In our retail data this would indicate that buying A has no effect on buying B.
- If lift > 1, A and B are positively correlated, so the 2 item sets lift the likelihood of each another. In our retail data this would indicate that customers tend to buy A and B together.
- If lift < 1, A and B are negatively correlated. In our retail data this would indicate that customers tend NOT to buy A and B together.
Coverage: The Coverage of the rule is the probability for the antecedent alone in the entire data set. Coverages tells how frequent your antecedent is in the data set. It thus gives an idea of how often the rule could be applied.
Step 7: Visualizing the association rules
Along with the arules library, the arulesViz library provides tools to visualize the rules generated by the apriori method. It can create static graphics or interactive html graphs that allow a user to scroll and zoom in the image. We can simply pass the rules object generated by the apriori method to the arulesViz::plot method, though in our case we have 450 rules which is far too many to fit into a single graph. Instead of passing all of the rules, we'll sort them by confidence and then graph the top 20.
top.conf <- sort(rules, decreasing = TRUE, na.last = NA, by = "confidence")
plot(top.conf[1:20], method="graph")
We can see that almost all of the high confidence rules contain whole milk in our dairy-loving data set.
Now, instead of sorting all of the rules by confidence, we'll sort them by lift and then graph the top 20.
top.lift <- sort(rules, decreasing = TRUE, na.last = NA, by = "lift")
plot(top.lift[1:20], method="graph")
Now we don't see a central cluster around whole milk, but instead a group of rules around sausages and frozen vegetables. This tells us that these are the items that might cause a shopper to buy another item. This helps give us a sense of the overall importance of specific items but also we can see from the lift, a sense of the importance of the rules as well.
Summary and next steps
In this tutorial, we've learned about the Apriori algorithm, an unsupervised machine learning algorithm that excels at association rule mining. We then learned to implement the Apriori algorithm in R using the arules library to analyze the Online Retail data set transactions and identify the relationships between items purchased together. We then visualized the resultant rules to show the relationships between specific items and the strength of those relationships.
Try watsonx for free
Build an AI strategy for your business on one collaborative AI and data platform called IBM watsonx, which brings together new generative AI capabilities, powered by foundation models, and traditional machine learning into a powerful platform spanning the AI lifecycle. With watsonx.ai, you can train, validate, tune, and deploy models with ease and build AI applications in a fraction of the time with a fraction of the data.
Try watsonx.ai, the next-generation studio for AI builders.
Next steps
Explore more articles and tutorials about watsonx on IBM Developer.