The Holy Trinity of ChatGPT, SBERT, and MapDB
Extending the experimental work on app. ChatGPT excels in generating human-like responses to textual inputs. On the other hand, SBERT is a state-of-the-art deep learning model for generating semantic sentence embeddings, which are used for various NLP tasks such as clustering and information retrieval. Finally, MapDB is a lightweight, high-performance database that provides concurrent access and off-heap storage capabilities.
Together, these three technologies can be a powerful combination in building large-scale, efficient, and intelligent applications that require advanced language processing and data management capabilities. For example, a chatbot built using ChatGPT and SBERT can provide highly personalized responses to users based on the context of their questions, while MapDB can be used to store and retrieve large amounts of data efficiently in a concurrent and fault-tolerant way.
Overall, the integration of ChatGPT, SBERT, and MapDB can lead to more efficient and intelligent applications that can handle complex and diverse data in a seamless and scalable way. Below diagram represents the basic connection between the systems:
The fundamental rules applied are:
- App only calls API,
- API goes to MapDB and collects the related questions,
- API then goes to sbert and finds most appropriate question that matches the user query,
- API goes to MAPDB and then checks if there is an answer to the question,
- If no answer is found in MapDB then ChatGPT is called.
- Then results are returned.
The code for SBERT server is here:
from flask import Flask, request, jsonify
from sentence_transformers import SentenceTransformer, util
import torch
from transformers import pipeline
app = Flask(__name__)
model = SentenceTransformer('multi-qa-MiniLM-L6-cos-v1')
fix_spelling = pipeline("text2text-generation",model="oliverguhr/spelling-correction-english-base")
query_embedding = model.encode('How big is London')
passage_embedding = model.encode(['London has 9,787,426 inhabitants at the 2011 census',
'London is known for its finacial district','How big is London','is large London'])
# Endpoint to create a new guide
@app.route('/sbertCos', methods=["POST"])
def sbertCos():
correctQuestion=fix_spelling(request.json['question'],max_length=1024)[0]['generated_text']
print(correctQuestion)
query_embedding = model.encode(correctQuestion)
passage_embedding = model.encode(request.json['questionArray'])
# Find the closest 5 sentences of the corpus for each query sentence based on cosine similarity
top_k = min(1, 1)
# We use cosine-similarity and torch.topk to find the highest 5 scores
cos_scores = util.cos_sim(query_embedding, passage_embedding)[0]
top_results = torch.topk(cos_scores, k=top_k)
top_results[1][0]
return request.json['questionArray'][top_results[1][0]]
app.run()Using SBERT on top of ChatGPT can indeed increase the speed of responses significantly. This is because SBERT allows for faster and more accurate semantic similarity matching, which is a key part of question-answering systems like the one described. By encoding questions and passages as high-dimensional vectors, SBERT can quickly compare the similarity between them using cosine similarity or other measures, allowing it to rapidly find the most relevant passage for a given query.
By using SBERT to filter the results from ChatGPT, the system can quickly identify the most relevant question and minimize the amount of computation required to generate a response. This can result in significant speed improvements, especially when the database of questions and answers is large.
However, it is important to note that SBERT is not a replacement for ChatGPT — rather, it is a complementary technology that can enhance the performance of question-answering systems. ChatGPT is still necessary to generate the actual responses, but by using SBERT to filter the input, the system can operate more efficiently and provide faster, more accurate answers to user queries.
Results
One might ask what users might be doing on the app, here is heuristics created with help of process mining for same:
Let’s zoom-in and see:
This intricate system is designed to optimize the process of finding the most appropriate response to user queries. By segregating responsibilities, it enables each component to perform its unique function with maximum efficiency. The app solely interacts with the API, which then utilizes MapDB to retrieve relevant questions. The API then employs sbert to find the most appropriate question that matches the user query. If no answer is found in MapDB, ChatGPT is called upon to provide a response. The complexity of this system is obscured from the user, as it operates seamlessly in the background to provide accurate and timely responses.
References
Engineering Apps with ChatGPT | True Geometry’s Blog
Engineering Apps | True Geometry’s Blog
Bending Stress calculation | True Geometry’s Blog
SentenceTransformers Documentation — Sentence-Transformers documentation (sbert.net)
