Implementing the Apriori algorithm in Python on watsonx.ai

The Apriori algorithm is a popular data mining technique to identify frequent item sets, which are a group of items that occur together in a data set. Item sets are important for identifying similarities across events, discrete classes, or categories. For examples, shoppers at a hardware store may buy both a hammer and nails or both paint and paintbrushes.

In this tutorial, you'll learn the fundamentals of implementing an Apriori algorithm in Python using Jupyter Notebooks on IBM watsonx.ai. We will use the Online Retail data set, which is a transnational data set containing transactions from a UK-based online retailer between December 1st, 2010 and December 9th, 2011. This is a multivariate, sequential, time-series data set suitable for classification and clustering tasks.

More about the Apriori algorithm

The Apriori algorithm runs iterations over the data to identify k-item-sets, meaning k items that frequently occur together. It then uses the k-item-sets to identify the k+1 item sets.

Apriori is an unsupervised machine learning algorithm that excels at association rule mining, a technique that identifies complex inter-relations between different variables of large data sets. The technique identifies frequent patterns, connections, and dependencies among candidate item sets.

The Apriori algorithm is based on the Apriori property which states that if an item set appears frequently in a data set then all of its subsets must also be frequent. Conversely, if an item set is identified as infrequent, all its supersets will also be considered infrequent.

The Apriori algorithm is widely used for data analysis, especially transactional data analysis. Let's look at a few use cases:

• When running a market basket analysis, where retail stores examine customer purchase history to identify items frequently bought together and then optimize shelf space by placing them closer together.

• Recommender systems can be improved by studying product-based relationships based on user preferences and purchase patterns.

• To identify fraudulent patterns in financial transactions, for instance, where particular combinations of items might indicate fraudelent activity.

• In web usage mining, we can analyze clickstream data and user behavior and then apply that information to improve recommender systems.

• In healthcare, Apriori can help dentify strong association rules between symptoms and diseases, enhancing the efficiency of diagnosis and devising targeted treatment plans.

• To help analyze data patterns in unsupervised learning-based artificial intelligence applications like clustering algorithms.

One of the most significant advantages of using the Apriori algorithm is its simplicity and adaptability. Apriori algorithms are not as efficient when handling large data sets because the multi-iteration process of item set candidate generation can become computationally expensive and memory-intensive. A combination of Apriori with other techniques is often used to mitigate these performance issues.

The Apriori algorithm function is well-established and integrated into many software libraries of popular programming languages like Python and Java. Its reliable implementation efficiently generates quality association rules and frequent item sets.

Prerequisites

You need an IBM Cloud account to create a watsonx.ai project.

Steps

We'll use the Apriori algorithm to analyze the transactions in this data set and identify relationships between items bought together. The steps include importing the libraries, loading the data set, exploring it, cleaning out the unnecessary values, preparing the data set into the required format, and then applying the Apriori algorithm. We'll also visualize the efficacy of the association rules evaluated by the model.

Step 1. Set up your environment

While you can choose from several tools, this tutorial walks you through how to set up an IBM account to use a Jupyter Notebook. Jupyter Notebooks are widely used within data science to combine code, text, images, and data visualizations to formulate a well-formed analysis.

1. Log in to watsonx.ai using your IBM Cloud account.

2. Create a watsonx.ai project.

3. Create a Jupyter Notebook using the Python Kernel.

This step will open a Notebook environment where you can load your data set and copy the code from this tutorial to perform the association rule mining process.

Step 2. Install and import libraries

We'll need to install the following libraries to complete this tutorial:

• pandas library to handle the data set.

• mlxtend library to implement the Apriori algorithm.

• NumPy to create arrays as required.

• Matplotlib to create visualizations.

Before moving forward, ensure you have installed all the required libraries on your console. If you haven't done so, install the libraries using the following commands.

#Install the required libraries
!pip install pandas
!pip install numpy
!pip install mlxtend
!pip install matplotlib

Next, you need to import the libraries. You need to import the pandas and NumPy libraries, the pyplot function from matplotlib, and the apriori and association_rules functions from mlxtend.frequent_patterns.

#Import the required libraries
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
from mlxtend.frequent_patterns import apriori, association_rules
%matplotlib inline

Step 3. Load the data set

In this step, you need to first download the Online Retail data set as a .zip file from the UCI Machine Learning Repository. Then, extract the Excel file that contains the data set.

#Download the data set
!wget https://archive.ics.uci.edu/static/public/352/online+retail.zip

#Extract the files
import zipfile
with zipfile.ZipFile('online+retail.zip', 'r') as datafilezip:
  print(datafilezip.infolist())
  datafilezip.extract("Online Retail.xlsx")

Once you extract the Excel file, you can load the data set as a pandas dataframe using the read_excel function of the pandas library.

We use the read_excel function in this step, as the data set we are working with is in the .xlsx format. However, data sets are more commonly available in the .csv format. For such files, you would use the read_csv function.

#Load the data set
df = pd.read_excel('Online Retail.xlsx')

Step 4. Explore the data set

In this step, we will explore the data set to understand the data structure and how you can interpret the data. We'll use the df.head()function to extract the first five entries of the pandas dataframe.

#Extract the first five entries
df.head()

Here, we see the initial entries of the pandas dataframe

Each row shows an item that was sold. For example, the first row shows that six pieces of "White Hanging Heart T-Light Holder," stored under the stock code 85123A and costing 2.55 GBP per unit, were sold to the customer bearing ID 17850 on December 1, 2010 at 8:26 AM, under the invoice number 536365 in the United Kingdom. The subsequent entries are also under the same invoice number, indicating they were all purchased together. This will be key to helping identify frequent item sets.

To access the list of column names only, you can use the df.columns function.

#Access the list of column names
colnames = df.columns
colnames

This returns all of the columns in the data set:

You can also view the countries from which this data has been compiled using the df.Country.unique() function. This information is crucial for purchase pattern analysis, as the purchasing patterns vary across American, European, Middle Eastern, and Asian countries.

#Extract the list of countries
df.Country.unique()

This gives us the list of countries as an array:

array(['United Kingdom', 'France', 'Australia', 'Netherlands', 'Germany',
       'Norway', 'EIRE', 'Switzerland', 'Spain', 'Poland', 'Portugal',
       'Italy', 'Belgium', 'Lithuania', 'Japan', 'Iceland',
       'Channel Islands', 'Denmark', 'Cyprus', 'Sweden', 'Austria',
       'Israel', 'Finland', 'Bahrain', 'Greece', 'Hong Kong', 'Singapore',
       'Lebanon', 'United Arab Emirates', 'Saudi Arabia',
       'Czech Republic', 'Canada', 'Unspecified', 'Brazil', 'USA',
       'European Community', 'Malta', 'RSA'], dtype=object)

Step 5. Clean the data set

You'll notice that the data set also contains entries for items returned to the store. The invoice numbers of these entries will begin with the letter "C" indicating that these are credit entries. You will also see a few entries for "Adjusting Bad Debt." The invoice number for such entries will begin with the letter "A." These entries are extraneous to our sales analysis so we'll clean the data set to exclude these entries from the dataframe.

First, let's check the length of the dataframe before and after eliminating these entries to verify the removal of unnecessary data.

#Length of the dataframe before cleaning
len(df)

From this you should see that we have a total of 541,909 entries in the dataframe.

Next, let's view the entries with invoice numbers beginning with "C." We'll need to first convert the data type of the invoice number attributed to a string and then search the string for a 'C'.

#Identify entries with "C" in the invoice number
df['InvoiceNo'] = df['InvoiceNo'].astype('str')
df[df['InvoiceNo'].str.contains('C')].head()

We have quite a few of them:

Note that all these invoice numbers begin with the letter "C." The output indicates that 9,288 credit entries must be removed from the data.

Next, let's look at the entries with invoice numbers beginning with "A."

#Identify entries with "A" in the invoice number
df[df['InvoiceNo'].str.contains('A')]

The output indicates that there are three entries for "Adjusting Bad Debt" that will have to be removed from the data. Observe that all these invoice numbers begin with "A." We'll need to remove the entries with a "C" or an "A" in the invoice number.

#Exclude unnecessary entries from the dataframe
df = df[~df['InvoiceNo'].str.contains('C|A')]
len(df) # this now prints 532618

The length of our dataframe has been reduced by 9,291 entries from 541,909 to 532,618. The updated dataframe "df" will now only contain entries reflecting a purchase.

Step 6. Prepare the data set

Now it's time to prepare the data set for the Apriori algorithm. We must organize the data to get a unique list of all purchased items. Additionally, we'll want these items to be hot encoded for each unique invoice number.

First, we'll create a subset dataframe, which only retains the entries recorded from a few European countries, for example, Denmark, Germany, and Italy. We'll then group the data based on the invoice number and item description while adding the quantity of each unique item.

#Grouping data for specific countries
subset_df = df[df['Country'].isin(['Denmark', 'Germany', 'Italy'])]
subset_df = subset_df.groupby(['InvoiceNo', 'Description'])['Quantity'].sum()
subset_df.head()

Now we have the data grouped by the invoice numbers and all products purchased under the invoice, with their quantities displayed. Such grouping enhances the readability of each invoice's specific details.

Next, we'll pivot the dataframe so that each unique entry in the "Description" becomes a column and each unique "InvoiceNo" becomes a row. This will allow us to shape the data so that we can pass all the items present in an invoice to the Apriori algorithm.

#Unpack the data
subset_df = subset_df.unstack().reset_index()
subset_df.head()

In this output, the data has been unpacked, with each unique entry of "Description" as as a column and each unique "InvoiceNo" depicted as a row. Based on the previous output, each unique "InvoiceNo--Description" pair will have their quantities reflected. All other entries will be NaN (not a number) by default.

Finally, we replace all the NaN values with zero and make the invoice number the dataframe's index.

#Replace the NaN values and make the invoice number the index
subset_df = subset_df.fillna(0)
subset_df = subset_df.set_index('InvoiceNo')
subset_df.head()

In this output table, all the "NaN" values have been replaced with "0," and the dataframe index has been modified to the unique invoice number associated with each row.

In the bottom-left corner of the output table, the column count of 1787 includes a column titled "POSTAGE." POSTAGE entries indicate that the items listed in the invoice have been booked to be sent via postal services. Since POSTAGE is a hidden column in the output table, we will print the contents of the column to verify its presence.

#View POSTAGE column
subset_df.head()['POSTAGE']

Since POSTAGE is not a product entry we will want to remove it from our dataframe before we pass it to the Apriori algorithm.

#Remove POSTAGE entries
subset_df = subset_df.drop("POSTAGE", axis=1)

We now have 1786 columns to represent our 1786 products.

One last step, though. The Apriori algorithm requires this data to be hot encoded, which means all entries with a non-zero value should be true and all entries with a zero value should be false. You can perform hot encoding by creating a simple encoding function and applying that as a map to the table.

#Encode the data
def hot_encode(x):
    if(x<= 0):
        return False
    else:
        return True

subset_df = subset_df.applymap(hot_encode)
subset_df.head()

Here we have the hot encoded dataframe for the Apriori algorithm:

After hot encoding, the generated dataframe has "True" for all entries with a non-zero value and "False" against all zero values. That is, if a given product description is part of a specific invoice number, the dataframe will have the value "True." Otherwise, the value will be "False."

In the output displayed in our example, we can only view the first five entries of the dataframe. There are 2101 columns in this dataframe, but only a few have been displayed. All the entries displayed here have "False" values. However, the entries for the respective items in each invoice number will be "True," though not visible in this output.

Step 7. Apply the Apriori algorithm to determine association rules in item sets

Now we will apply the Apriori algorithm to the prepared dataframe to identify frequent item purchases by calling apriori. The Apriori function reads the hot encoded data and determines the support parameter for each of the unique product descriptions, along with the item sets that the model creates. The model iteratively processes the data, identifying all the frequent item sets, filtering them based on the minimum support threshold, and then appending them to an output dataframe. The higher the support value, the more frequent the purchase of the item set.

An item is considered frequent based on a parameter called "support." The "support" of an item (or item set) is the ratio of the number of times an item occurs in different transactions.

You can indicate a required minimum support threshold when applying the Apriori algorithm. This means that any item or item set with "support" less than the specified minimum support will be considered infrequent. You can set a minimum support threshold using the argument min_support, which will filter out the less frequently purchased items. The smaller the minimum support threshold value, the more confidently the algorithm will identify relations. For this tutorial, we'll set the min_support value at "0.05."

#Apply the Apriori algorithm
frq_items = apriori(subset_df, min_support=0.05, use_colnames=True)
frq_items.sort_values(['support'], ascending=[False])

Here, we can see the frequent items and item sets generated by the Apriori algorithm:

The output of the Apriori function has been sorted in descending order by "support" of each of the item sets.

This set of frequent items can now be used for association rule learning based on their different combinations.

The next step is association rule mining. We take the output of the Apriori algorithm and learn the association rules for the item sets. For this tutorial, we have chosen the lift parameter as the metric of association. Lift quantifies the strength of association between the two items. It assesses whether there is enough evidence of a real relationship between the items or if their co-occurrence is merely coincidental.

Let's say we have two items, A and B. The lift of the antecedent 'A' to the consequent 'B' is calculated as Support(A ∪ B) / (Support(A) * Support(B)). This could be described as "the support of A given B divided by the support for A times the support for B". We interpret the values like this:

• If lift = 1, A and B are completely independent. In our retail data this would indicate that buying A has no effect on buying B.
• If lift > 1, A and B are positively correlated, so the 2 item sets lift the likelihood of each another. In our retail data this would indicate that customers tend to buy A and B together.
• If lift < 1, A and B are negatively correlated. In our retail data this would indicate that customers tend NOT to buy A and B together.

We'll set the minimum threshold at 1. As 1 represents the minimum possible value for the lift parameter, it implies that we are generating all possible association rules based on the frequent item sets generated by the Apriori algorithm.

rules=association_rules(frq_items, metric='lift', min_threshold=1)
rules.sort_values(['lift'], ascending=[False])

This gives us the association rules sorted with respect to the lift parameter:

The output displays the association between the different item sets. Here, "antecedents" represent the items likely to be picked up first, and "consequents" represent the items picked up because of that choice.

Based on the output of the Apriori function, the "support" value for both antecedents and consequents is also displayed. The support parameter represents the frequency of the two item sets together. On the other hand, the confidence parameter represents the ratio of the number of transactions involving both the antecedent and the consequent to the number of transactions with only the antecedent. In short, the confidence parameter represents the likelihood of the consequent being picked up, provided the antecedent is picked up.

The lift parameter indicates the ratio of "confidence" to the "antecedent support," indicating the joint likelihood of the antecedent and the consequent being picked up together.

Other parameters, such as leverage, conviction, and zhangs_metric, are also derived using the support, confidence, and lift values.

We'll want to sort the results based on the lift parameter, as it helps understand the quality of the association rule determined from the data. The comparison of the values of the confidence and lift parameters is an excellent criterion to determine if the association rule determined here is strong.

We can take a closer look at the second entry in this output to understand the determined association rules. There is approximately 86% chance of the "WOODLAND CHARLOTTE BAG" being picked up, provided the "RED RETROSPOT CHARLOTTE BAG" is purchased first. This likelihood is also substantiated by the high value of the lift parameter (7.208), indicating that the two are often purchased together.

Similarly, observation of the entry against index 14 suggests there is approximately 82% chance of "ROUND SNACK BOXES SET OF 4 WOODLAND" being picked up if "ROUND SNACK BOXES SET OF 4 FRUITS" is picked up first. However, the low lift parameter value (3.440) suggests less evidence to substantiate this association rule.

Step 8. Evaluation of the result

Finally, let us examine the quality of the association rules generated by comparing the confidence and lift parameters for a random subset. We will extract the lift and confidence values of 10 random samples from previously generated rules. We'll then normalize both entities with respect to their maximum values and format them to be plotted against each other for comparison. We will create a clustered column graph to compare the results.

#Random sampling of association rules for comparison of confidence and lift values
rules_random=rules.sample(10, random_state = 42)
rules_lift = rules_random[['lift']].to_numpy()
rules_lift = (rules_lift/rules_lift.max()).transpose()[0]
rules_conf = rules_random[['confidence']].to_numpy()
rules_conf = (rules_conf/rules_conf.max()).transpose()[0]
width = 0.40
plt.figure(figsize=(12, 6), dpi=200)

# plot data in grouped manner of bar type
plt.bar(np.arange(len(rules_random))-0.2,rules_lift, width, color='black')
plt.bar(np.arange(len(rules_random))+0.2,rules_conf, width, hatch='//', edgecolor='black', facecolor='white')
plt.xlabel('Instance index')
plt.ylabel('Normalized metric value')
plt.legend(['lift','confidence'])
plt.xticks(range(0,10));

In the bar graph, the numbers on the x-axis represent the IDs assigned to the random samples, from 0 to 9. If the confidence value and the lift value appear to be on a similar scale, it can be concluded that there is sufficient evidence to support the specific association rule. However, if the confidence value is much higher than the lift value, there is little evidence to support that association rule. A higher lift value with a low confidence value would indicate a reversal of choice for antecedents and consequents.

This graph is generated by choosing a set of random samples from the data. Therefore, on execution, the code might generate a different-looking graph each time.

So, when both the confidence and lift parameters are relatively high, as observed for items 0, 4, and 7, it indicates that the association rule developed between the antecedent and consequent is strong and sufficiently established. Similarly, when both the parameters are low, as observed for items 1 and 6, it indicates that the likelihood of the two being picked up together is relatively low, and the same is also appropriately evidenced by the data. However, the high confidence and low lift values for items 8 and 9 indicate an association rule not sufficiently proven by the data and hence can be considered lower quality. Also, note that the high lift value and the low confidence value for item 3 point to a possible reversal of antecedents and consequents.

Alternative implementations of the Apriori algorithms

There are other ways in which Apriori algorithms can be implemented in Python programming. For example, there is a different library called apyori that hosts a function apriori for the same algorithm. This function's usage differs and requires the data to be formatted differently.

#Alternative library for the Apriori algorithm
from apyori import apriori

This function directly generates association rules based on preprocessed data using additional parameters like min_support (minimum support of the item set), min_confidence (minimum threshold value of confidence for the association rule), min_lift (minimum threshold for the lift parameter of the association rule), and min_length (minimum length of the item set proposed).

You can also use Python libraries like scikit-learn, a popular repository for various machine learning algorithms, to create a custom Apriori algorithm from scratch.

Summary

In this tutorial, you learned that Apriori is an unsupervised machine learning algorithm that excels at association rule mining. You then learned to implement the Apriori algorithm to analyze the Online Retail data set transactions and identify the relationships between items purchased together. Apriori analysis is typically used to generate recommendations for associated item sets.

Try watsonx for free

Build an AI strategy for your business on one collaborative AI and data platform called IBM watsonx, which brings together new generative AI capabilities, powered by foundation models, and traditional machine learning into a powerful platform spanning the AI lifecycle. With watsonx.ai, you can train, validate, tune, and deploy models with ease and build AI applications in a fraction of the time with a fraction of the data.

Try watsonx.ai, the next-generation studio for AI builders.

Next steps

Explore more articles and tutorials about watsonx on IBM Developer.