🏫ESP32 ChatGLM for Custom Conversations - 中文文档
Technological achievements from Tsinghua University ---> "ChatGLM" 🇨🇳 on Espressif's ESP32 platform by Arduino!
| Number | Parts | Description |
|---|---|---|
| 1 | Name | 📼 ESP32 ChatGLM |
| 2 | Version | 🕸 Version 0.0.5 |
| 3 | Support Audio and Microphone | 📢 Option (Yes) |
| 4 | Support Web | 🎭 Yes |
| 5 | Support Mobile | 📱 No |
Use the official asynchronous calling method to request its API. The API is obtained from the following websites:
📎 API receive : https://open.bigmodel.cn/
Download Arduino IDE and install it. Open the IDE and find the File -> Perference.
Please Use ESP32 Addritional boards manager URLs.
Download the Project and Download libraries. (Build in just use it, otherwise download it) These are libraries for this project:
#include <Arduino.h> //build in
#include <CustomJWT.h> //find from library
#include <ESPAsyncWebServer.h> // Get from https://github.com/me-no-dev/ESPAsyncWebServer
#include <ArduinoJson.h> //find from library
#include <WiFiClientSecure.h> //build in
#include <WiFiUdp.h> //build in
#include <HTTPClient.h> //build in
#include <NTPClient.h> //find from library
#include <queue> //build in
#include <SPIFFS.h> //build in
#include <SPI.h> //build in
#include <SoftwareSerial.h> //find from library
#include "UTF8ToGB2312.h" //find from library
#include <esp_heap_caps.h> //build in
#include "base64.h" //build in
FInd the library folder and find the CustomJWT Folder. (I also upload CustomJWT.h file, Directly replace with me CustomJWT ☝️)
Find the CustomJWT.h file in src folder. to add some code in CustomJWT(........) (in brackets)
char *sign_type = "SIGN" add under char *typ = "JWT" for each CustomJWT method!
IMPORTANT!
there got a problem for Library JWT library(The Library name called: CustomJWT)
sprintf(headerJSON, "{\"alg\": \"%s\",\"typ\":\"%s\",\"sign_type\":\"%s\"}", alg, typ, sign_type);
Error: The is a problem between "alg": "%s"which has a space between "alg": and "%s"! Please delete it! if not it will show different Base64 code.
sprintf(headerJSON, "{\"alg\":\"%s\",\"typ\":\"%s\",\"sign_type\":\"%s\"}", alg, typ, sign_type);
Place the ChatGLM.ino and index.h projects into your own project folder with the name: ChatGLM (here the folder name needs to be the same as the *.ino file). You can change your personal Request API and WiFI, even change the role content of system_role and NTP network time server, and you can even add a new index.h file to index.h.
Connect the ESP32 Module by Serial Port and PC's USB. Please choose correct ESP32 board to PC, Mine is YD-ESP32-S3-N16R8(v1.3)
And my Arduino Setting is：
Finally Happy Flashing your ESP32-S3 device! 😄🥇
The programme use Max9814 Microphone and SNR9816VR_TTS Module which you can find in Taobao (China) and Alibaba(Global)
Connect all Module together:
#define MYPORT_TX 0 //Connect SNR9816VR_TTS RX_PIN and Connect ESP32-S3 PIN_0
#define MYPORT_RX 1 //Connect SNR9816VR_TTS TX_PIN and Connect ESP32-S3 PIN_RX
#define audio_out 4 //Connect ESP32-S3 PIN_4 with MAX9814 OUT
#define audio_keys 6 //Connect ESP32-S3 PIN_6 with button(Default high level)
#define disk_format 7 //Connect ESP32-S3 PIN_7 with button(Default high level)
If you flow AR that, it's gain 60db of Voice
If you connect AR to Ground, it's gain 50db of Voice <--- I choose
If you connect AR to 3.3V, it's gain 40 db of Voice
This code is an interrupt servicer that handles timer interrupts. The logic for ADC data acquisition is implemented in the timer interrupt service routine. When a condition is met (e.g. audio_start_flag is set to 1), the interrupt service routine reads data from audio_out and stores it until the acquisition is complete.
void IRAM_ATTR onHardWareTimer() {
portENTER_CRITICAL_ISR(&timerMux);
if (audio_start_flag == 1) {
audio_output[var_num] = analogRead(audio_out);
var_num++;
if (var_num >= data_len) {
audio_complete_flag = 1;
audio_start_flag = 0;
var_num = 0;
}
}
portEXIT_CRITICAL_ISR(&timerMux);
}
If you want to know this part, you can open my project to find it <ChatGLM.ino>.
This Module is only support GBK2312 for Chinese (Mine is not support English, you may bought English version of this board)
Base on picture shows that you need 5V to drive the board but ESP32 S3 for me doesn't support 5V (ESP32C3 got 5V output):
This function converts the string into GB2312 encoded format and sends it to the serial device.
void speech(String data) {
String gb2312_str = GB.get(data);
unsigned int dat_len = gb2312_str.length() + 3;
unsigned char head[dat_len + 3];
head[0] = 0xFD; // head byte
head[1] = dat_len >> 8; // length byte 1
head[2] = dat_len; // length byte 2
head[3] = 0x01; // cmd byte
head[4] = 0x01; // para byte
for (int i = 0; i < gb2312_str.length(); i++) {
head[i + 5] = gb2312_str[i];
}
unsigned char xor_value = head[0];
for (int i = 1; i < dat_len + 2; i++) { // Iterates through the header array (except for the header start flag), performs a different-or operation, and computes the checksum value.
xor_value ^= head[i]; // Store the checksum value in the last byte of the header
}
head[dat_len + 2] = xor_value;
myPort.write(head, dat_len + 3);
delay(gb2312_str.length() * 10); // Delayed for a period of time to ensure that the receiver has enough time to process the received data. The delay here is determined by the length of the GB2312 string
}
Currently supports ChatGLM-4 models, if you need ChatGLM-3-Turbo models you can download this project and modify it in utiltools.h file. AI roleplaying is also currently integrated. By default, it uses asynchronous calls and non-role-playing prompt, if you have other questions, you can handin a Discussion or start an Issue and I will answer your questions if I see them as soon as possible! You are also very welcome to fork this project, and if you don't mind you can also give my project a ⭐! Thanks!
Last thanks to @JoinChang and other libraries providers 👍
