An alternative to multi-selct fields and using AI to obtain Keywords

Recently, I needed to categorise pages on a website according to service offerings and compile a list of keywords for each page. Luckily, I already had a list of the pages in Tape, so I just had to add the page text and set up some automation.

I opted to use AI (specifically GPT) for the task of extracting keywords and assigning categories. The categories were relatively straightforward - a choice of six. I provided the webpage text and the six options, then instructed the AI to select the relevant categories (which could be multiple) and input the results into a multi-select field.

melotte.consulting 2023-10-30 at 16.14.46

However, handling the categories was a bit tricky. I wasn’t sure what the AI would come back with, and unfortunately, you can’t create multi-select field options on the fly via automation. So, I needed a different approach. I came up with the idea of creating a separate app to store all the Keywords and then establishing a two-way relationship with the relevant web pages.

Next, I wanted to guide the AI to use existing Keywords whenever possible. I didn’t want redundant entries like ‘Tape API’ and ‘API’. I simply wanted ‘API’. To achieve this, I searched within the Keywords app with no filters and used the following code to extract the name and format it appropriately for sending with the prompt:

kw = jsonata('title').evaluate(record_collection_keywords)
var_currentkeywords = JSON.stringify(kw);

I’ll admit, I’m still fine-tuning the prompt for maximum benefit, but it’s proving to be quite useful.

From there, it’s just a matter of sending the ‘Current Keywords’, webpage text, and prompt to OpenAI via their API and waiting for the results. When the response comes back, I parse out the new Keywords and categories:

const { category, keywords } = json_response;

Now, here’s where things could be a bit more streamlined, but for now, this method has been effective and I haven’t had the opportunity to explore improvements yet. I input the categories into a text field and then run a series of six conditionals. If a keyword exists in the text field, then the relevant option is added to the multi-select field.

melotte.consulting 2023-10-30 at 16.18.37

Moving on to the new Keywords, I wanted to ensure I had clean and correct collections to work with. So, I cleared the previous collections:

melotte.consulting 2023-10-30 at 16.18.14

Then, it’s a matter of looping through each new Keyword and searching for it in the existing Keywords. If it’s found, I establish a two-way relationship. If it’s not, I create it and then establish the relationship before continuing with the loop.

Now, I have a comprehensive list of keywords and their relation to the web pages, along with the number of pages they’re associated with. In my opinion, this approach offers more flexibility for future processing and analysis compared to using a multi-select field.

One more thing worth mentioning is that I ended up instructing the AI to provide a maximum of the 10 most relevant Keywords per page. This seemed to work better for my purposes than giving it free rein and having it return 20 per page.

I have made a quick video which may explain the process better:

6 Likes

While I don’t currently have a personal use for this at the moment, I wanted to express my appreciation for your openness in notating your process, and your efforts to ensure that valuable information is accessible to the community without any gatekeeping. Your generosity is truly commendable. We’re lucky to have you here!

5 Likes

Hi @Jason,

your showcase and especially your implementation is really completely outstanding! :rocket: :trophy:
I think this could be very helpful for many users and your detailed step by step explanation including video tutorial makes it very accessible to build it.

Many many thanks for this extraordinary showcase :100:

Cheers
Leo

4 Likes