How to set IMAP ID (RFC2971)

[dingxiao88]

Build tool used:

• Arudino
  Platformio (Visual Studio Code plugin)
  Platformio CLI
  Other

Board used (ESP32/ESP8266/Arudino):

ESP8266-based

• ESP32-based

Other Libraries That are used:

Description of problem:

Hello, I am using your library on ESP32 to receive emails via the IMAP service. When selecting the email directory, the email service provider prompts me with an "EXAMINE Unsafe Login" error. After consulting with them, they informed me that "the third-party client was judged as an unsafe login because it did not carry the IMAP ID", which is an extended IMAP command defined in RFC2971.

RFC2971_
something like this：

id_(parameters=None)[source]
Issue the ID command, returning a dict of server implementation fields.
parameters should be specified as a dictionary of field/value pairs, for example: {"name": "IMAPClient", "version": "0.12"}

python code：

from imapclient import IMAPClient
server = IMAPClient("imap.163.com", ssl=True, port=143)
server.login("<user>", "<passwd>")
server.id_({"name": "IMAPClient", "version": "2.1.0"})
messages = server.select_folder('INBOX')

Now I do not know how to add it to my application. I hope you can help me with this. Thank you very much.

Share code snippet to reproduce the issue:

PASTE .cpp / .ino code here


#include <Arduino.h>

#define ESP32

#if defined(ESP32) || defined(ARDUINO_RASPBERRY_PI_PICO_W)
#include <WiFi.h>
#elif defined(ESP8266)
#include <ESP8266WiFi.h>
#else

// Other Client defined here
// To use custom Client, define ENABLE_CUSTOM_CLIENT in  src/ESP_Mail_FS.h.
// See the example Custom_Client.ino for how to use.

#endif

#include <ESP_Mail_Client.h>

// Provide the SD card interfaces setting and mounting
#include <extras/SDHelper.h>

#define WIFI_SSID "ssid"
#define WIFI_PASSWORD "password"

/** For Gmail, IMAP option should be enabled. https://support.google.com/mail/answer/7126229?hl=en
 * and also https://accounts.google.com/b/0/DisplayUnlockCaptcha
 *
 * Some Gmail user still not able to sign in using account password even above options were set up,
 * for this case, use "App Password" to sign in instead.
 * About Gmail "App Password", go to https://support.google.com/accounts/answer/185833?hl=en
 *
 * For Yahoo mail, log in to your yahoo mail in web browser and generate app password by go to
 * https://login.yahoo.com/account/security/app-passwords/add/confirm?src=noSrc
 *
 * To use Gmai and Yahoo's App Password to sign in, define the AUTHOR_PASSWORD with your App Password
 * and AUTHOR_EMAIL with your account email.
 */

/* The imap host name e.g. imap.gmail.com for GMail or outlook.office365.com for Outlook */
#define IMAP_HOST "imap.163.com"

/** The imap port e.g.
 * 143  or esp_mail_imap_port_143
 * 993 or esp_mail_imap_port_993
 */
#define IMAP_PORT 143

/* The log in credentials */
#define AUTHOR_EMAIL "email"
#define AUTHOR_PASSWORD "pass"

/* Callback function to get the Email reading status */
void imapCallback(IMAP_Status status);

/* Print the list of mailbox folders */
void printAllMailboxesInfo(IMAPSession &imap);

/* Print the selected folder info */
void printSelectedMailboxInfo(SelectedFolderInfo sFolder);

/* Print all messages from the message list */
void printMessages(MB_VECTOR<IMAP_MSG_Item> &msgItems, bool headerOnly);

/* Print all attachments info from the message */
void printAttacements(MB_VECTOR<IMAP_Attach_Item> &atts);

/* Declare the global used IMAPSession object for IMAP transport */
IMAPSession imap;

#if defined(ARDUINO_RASPBERRY_PI_PICO_W)
WiFiMulti multi;
#endif

void setup()
{

    Serial.begin(115200);

#if defined(ARDUINO_ARCH_SAMD)
    while (!Serial)
        ;
    Serial.println();
    Serial.println("**** Custom built WiFiNINA firmware need to be installed.****\n");
    Serial.println("To install firmware, read the instruction here, https://github.com/mobizt/ESP-Mail-Client#install-custom-build-wifinina-firmware");
#endif

    Serial.println();

#if defined(ARDUINO_RASPBERRY_PI_PICO_W)
    multi.addAP(WIFI_SSID, WIFI_PASSWORD);
    multi.run();
#else
    WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
#endif

    Serial.print("Connecting to Wi-Fi");
    unsigned long ms = millis();
    while (WiFi.status() != WL_CONNECTED)
    {
        Serial.print(".");
        delay(300);
#if defined(ARDUINO_RASPBERRY_PI_PICO_W)
        if (millis() - ms > 10000)
            break;
#endif
    }
    Serial.println();
    Serial.print("Connected with IP: ");
    Serial.println(WiFi.localIP());
    Serial.println();

    /*  Set the network reconnection option */
    MailClient.networkReconnect(true);

    // The WiFi credentials are required for Pico W
    // due to it does not have reconnect feature.
#if defined(ARDUINO_RASPBERRY_PI_PICO_W)
    MailClient.clearAP();
    MailClient.addAP(WIFI_SSID, WIFI_PASSWORD);
#endif

#if defined(ESP_MAIL_DEFAULT_SD_FS) // defined in src/ESP_Mail_FS.h
    // Mount SD card.
    SD_Card_Mounting(); // See src/extras/SDHelper.h
#endif

    /** Enable the debug via Serial port
     * 0 for no debugging
     * 1 for basic level debugging
     *
     * Debug port can be changed via ESP_MAIL_DEFAULT_DEBUG_PORT in ESP_Mail_FS.h
     */
    imap.debug(1);

    /* Set the callback function to get the reading results */
    imap.callback(imapCallback);

    /** In case the SD card/adapter was used for the file storagge, the SPI pins can be configure from
     * MailClient.sdBegin function which may be different for ESP32 and ESP8266
     * For ESP32, assign all of SPI pins
     * MailClient.sdBegin(14,2,15,13)
     * Which SCK = 14, MISO = 2, MOSI = 15 and SS = 13
     * And for ESP8266, assign the CS pins of SPI port
     * MailClient.sdBegin(15)
     * Which pin 15 is the CS pin of SD card adapter
     */

    /* Declare the Session_Config for user defined session credentials */
    Session_Config config;

    /* Set the session config */
    config.server.host_name = IMAP_HOST;
    config.server.port = IMAP_PORT;
    config.login.email = AUTHOR_EMAIL;
    config.login.password = AUTHOR_PASSWORD;

    /** Declare the IMAP_Data object used for user defined IMAP operating options
     * and contains the IMAP operating result
     */
    IMAP_Data imap_data;

    /* Set the storage to save the downloaded files and attachments */
    imap_data.storage.saved_path = F("/email_data");

    /** The file storage type e.g.
     * esp_mail_file_storage_type_none,
     * esp_mail_file_storage_type_flash, and
     * esp_mail_file_storage_type_sd
     */
    imap_data.storage.type = esp_mail_file_storage_type_sd;

    /** Set to download heades, text and html messaeges,
     * attachments and inline images respectively.
     */
    imap_data.download.header = true;
    imap_data.download.text = true;
    imap_data.download.html = true;
    imap_data.download.attachment = true;
    imap_data.download.inlineImg = true;

    /** Set to enable the results i.e. html and text messaeges
     * which the content stored in the IMAPSession object is limited
     * by the option imap_data.limit.msg_size.
     * The whole message can be download through imap_data.download.text
     * or imap_data.download.html which not depends on these enable options.
     */
    imap_data.enable.html = true;
    imap_data.enable.text = true;

    /* Set to enable the sort the result by message UID in the decending order */
    imap_data.enable.recent_sort = true;

    /* Set to report the download progress via the default serial port */
    imap_data.enable.download_status = true;

    /* Header fields parsing is case insensitive by default to avoid uppercase header in some server e.g. iCloud
    , to allow case sensitive parse, uncomment below line*/
    // imap_data.enable.header_case_sensitive = true;

    /* Set the limit of number of messages in the search results */
    imap_data.limit.search = 5;

    /** Set the maximum size of message stored in
     * IMAPSession object in byte
     */
    imap_data.limit.msg_size = 512;

    /** Set the maximum attachments and inline images files size
     * that can be downloaded in byte.
     *"name": "IMAPClient", "version": "2.1.0" The file which its size is largger than this limit may be saved
     * as truncated file.
     */
    imap_data.limit.attachment_size = 1024 * 1024 * 5;

    //@-dx add -C:\Users\DX\Documents\Arduino\libraries\ESP_Mail_Client\src\ESP_Mail_Const.h
    // imap_data.dx_163.dx_163_cmd = F("/\"name\": \"IMAPClient\", \"version\": \"2.1.0\", \"vendor\": \"dx\");

    /* Connect to the server */
    if (!imap.connect(&config, &imap_data))
        return;

    if (imap.isAuthenticated())
        Serial.println("Successfully logged in.");
    else
        Serial.println("Connected with no Auth.");

    /*  {Optional} */
    printAllMailboxesInfo(imap);

    /* Open or select the mailbox folder to read or search the message */
    if (!imap.selectFolder(F("INBOX")))
        return;

    /*  {Optional} */
    printSelectedMailboxInfo(imap.selectedFolder());
    
    /** Message UID to fetch or read e.g. 100.
     * In this case we will get the UID from the max message number (lastest message)
     */
    imap_data.fetch.uid = imap.getUID(imap.selectedFolder().msgCount());

    // or fetch via the message sequence number
    // imap_data.fetch.number = imap.selectedFolder().msgCount();

    // if both imap_data.fetch.uid and imap_data.fetch.number were set,
    // then total 2 messages will be fetched i.e. one using uid and other using number.

    /* Set seen flag */

    // The message with "Seen" flagged means the message was already read or seen by user.
    // The default value of this option is set to false.
    // If you want to set the message flag as "Seen", set this option to true.
    // If this option is false, the message flag was unchanged.
    // To set or remove flag from message, see Set_Flags.ino example.

    // imap_data.fetch.set_seen = true;

    /* Read or search the Email and close the session */

    // When message was fetched or read, the /Seen flag will not set or message remained in unseen or unread status,
    // as this is the purpose of library (not UI application), user can set the message status as read by set \Seen flag
    // to message, see the Set_Flags.ino example.
    MailClient.readMail(&imap);

    /* Clear all stored data in IMAPSession object */
    imap.empty();
}

void loop()
{
}


/* Callback function to get the Email reading status */
void imapCallback(IMAP_Status status)
{
    /* Print the current status */
    Serial.println(status.info());

    /* Show the result when reading finished */
    if (status.success())
    {
        /* Print the result */
        /* Get the message list from the message list data */
        IMAP_MSG_List msgList = imap.data();
        printMessages(msgList.msgItems, imap.headerOnly());

        /* Clear all stored data in IMAPSession object */
        imap.empty();
    }
}

void printAllMailboxesInfo(IMAPSession &imap)
{
    /* Declare the folder collection class to get the list of mailbox folders */
    FoldersCollection folders;

    /* Get the mailbox folders */
    if (imap.getFolders(folders))
    {
        for (size_t i = 0; i < folders.size(); i++)
        {
            /* Iterate each folder info using the  folder info item data */
            FolderInfo folderInfo = folders.info(i);
            ESP_MAIL_PRINTF("%s%s%s", i == 0 ? "\nAvailable folders: " : ", ", folderInfo.name, i == folders.size() - 1 ? "\n" : "");
        }
    }
}

void printSelectedMailboxInfo(SelectedFolderInfo sFolder)
{
    /* Show the mailbox info */
    ESP_MAIL_PRINTF("\nInfo of the selected folder\nTotal Messages: %d\n", sFolder.msgCount());
    ESP_MAIL_PRINTF("Predicted next UID: %d\n", sFolder.nextUID());
    ESP_MAIL_PRINTF("Unseen Message Index: %d\n", sFolder.unseenIndex());
    for (size_t i = 0; i < sFolder.flagCount(); i++)
        ESP_MAIL_PRINTF("%s%s%s", i == 0 ? "Flags: " : ", ", sFolder.flag(i).c_str(), i == sFolder.flagCount() - 1 ? "\n" : "");
}

void printAttacements(MB_VECTOR<IMAP_Attach_Item> &atts)
{
    ESP_MAIL_PRINTF("Attachment: %d file(s)\n****************************\n", atts.size());
    for (size_t j = 0; j < atts.size(); j++)
    {
        IMAP_Attach_Item att = atts[j];
        /** att.type can be
         * esp_mail_att_type_none or 0
         * esp_mail_att_type_attachment or 1
         * esp_mail_att_type_inline or 2
         */
        ESP_MAIL_PRINTF("%d. Filename: %s, Name: %s, Size: %d, MIME: %s, Type: %s, Description: %s, Creation Date: %s\n", j + 1, att.filename, att.name, att.size, att.mime, att.type == esp_mail_att_type_attachment ? "attachment" : "inline", att.description, att.creationDate);
    }
    Serial.println();
}

void printMessages(MB_VECTOR<IMAP_MSG_Item> &msgItems, bool headerOnly)
{

    /** In devices other than ESP8266 and ESP32, if SD card was chosen as filestorage and
     * the standard SD.h library included in ESP_Mail_FS.h, files will be renamed due to long filename
     * (> 13 characters) is not support in the SD.h library.
     * To show how its original file name, use imap.fileList().
     */
    // Serial.println(imap.fileList());

    for (size_t i = 0; i < msgItems.size(); i++)
    {

        /* Iterate to get each message data through the message item data */
        IMAP_MSG_Item msg = msgItems[i];

        Serial.println("****************************");
        ESP_MAIL_PRINTF("Number: %d\n", msg.msgNo);
        ESP_MAIL_PRINTF("UID: %d\n", msg.UID);
        ESP_MAIL_PRINTF("Messsage-ID: %s\n", msg.ID);

        ESP_MAIL_PRINTF("Flags: %s\n", msg.flags);

        // The attachment status in search may be true in case the "multipart/mixed"
        // content type header was set with no real attachtment included.
        ESP_MAIL_PRINTF("Attachment: %s\n", msg.hasAttachment ? "yes" : "no");

        if (strlen(msg.acceptLang))
            ESP_MAIL_PRINTF("Accept Language: %s\n", msg.acceptLang);
        if (strlen(msg.contentLang))
            ESP_MAIL_PRINTF("Content Language: %s\n", msg.contentLang);
        if (strlen(msg.from))
            ESP_MAIL_PRINTF("From: %s\n", msg.from);
        if (strlen(msg.sender))
            ESP_MAIL_PRINTF("Sender: %s\n", msg.sender);
        if (strlen(msg.to))
            ESP_MAIL_PRINTF("To: %s\n", msg.to);
        if (strlen(msg.cc))
            ESP_MAIL_PRINTF("CC: %s\n", msg.cc);
        if (strlen(msg.date))
        {
            ESP_MAIL_PRINTF("Date: %s\n", msg.date);
            ESP_MAIL_PRINTF("Timestamp: %d\n", (int)MailClient.Time.getTimestamp(msg.date));
        }
        if (strlen(msg.subject))
            ESP_MAIL_PRINTF("Subject: %s\n", msg.subject);
        if (strlen(msg.reply_to))
            ESP_MAIL_PRINTF("Reply-To: %s\n", msg.reply_to);
        if (strlen(msg.return_path))
            ESP_MAIL_PRINTF("Return-Path: %s\n", msg.return_path);
        if (strlen(msg.in_reply_to))
            ESP_MAIL_PRINTF("In-Reply-To: %s\n", msg.in_reply_to);
        if (strlen(msg.references))
            ESP_MAIL_PRINTF("References: %s\n", msg.references);
        if (strlen(msg.comments))
            ESP_MAIL_PRINTF("Comments: %s\n", msg.comments);
        if (strlen(msg.keywords))
            ESP_MAIL_PRINTF("Keywords: %s\n", msg.keywords);

        /* If the result contains the message info (Fetch mode) */
        if (!headerOnly)
        {
            if (strlen(msg.text.content))
                ESP_MAIL_PRINTF("Text Message: %s\n", msg.text.content);
            if (strlen(msg.text.charSet))
                ESP_MAIL_PRINTF("Text Message Charset: %s\n", msg.text.charSet);
            if (strlen(msg.text.transfer_encoding))
                ESP_MAIL_PRINTF("Text Message Transfer Encoding: %s\n", msg.text.transfer_encoding);
            if (strlen(msg.html.content))
                ESP_MAIL_PRINTF("HTML Message: %s\n", msg.html.content);
            if (strlen(msg.html.charSet))
                ESP_MAIL_PRINTF("HTML Message Charset: %s\n", msg.html.charSet);
            if (strlen(msg.html.transfer_encoding))
                ESP_MAIL_PRINTF("HTML Message Transfer Encoding: %s\n\n", msg.html.transfer_encoding);

            if (msg.rfc822.size() > 0)
            {
                ESP_MAIL_PRINTF("\r\nRFC822 Messages: %d message(s)\n****************************\n", msg.rfc822.size());
                printMessages(msg.rfc822, headerOnly);
            }

            if (msg.attachments.size() > 0)
                printAttacements(msg.attachments);
        }

        Serial.println();
    }
}



**Additional information and things you've tried:**

21:00:24.107 -> ets Jun 8 2016 00:22:57
21:00:24.107 ->
21:00:24.107 -> rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
21:00:24.107 -> configsip: 0, SPIWP:0xee
21:00:24.107 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
21:00:24.160 -> mode:DIO, clock div:1
21:00:24.160 -> load:0x3fff0018,len:4
21:00:24.160 -> load:0x3fff001c,len:1216
21:00:24.160 -> ho 0 tail 12 room 4
21:00:24.160 -> load:0x40078000,len:10944
21:00:24.160 -> load:0x40080400,len:6388
21:00:24.160 -> entry 0x400806b4
21:00:24.460 ->
21:00:24.607 -> Connecting to Wi-Fi........
21:00:26.971 -> Connected with IP: 192.168.43.66
21:00:26.971 ->
21:00:26.971 ->
21:00:26.971 -> Mounting SD Card... failed
21:00:27.372 ->
21:00:27.372 -> Connecting to IMAP server...
21:00:27.372 -> > C: ESP Mail Client v3.0.3
21:00:27.372 -> > C: Connect to IMAP server
21:00:27.372 -> > C: Host > imap.163.com
21:00:27.372 -> > C: Port > 143
21:00:27.773 -> IMAP server connected
21:00:27.773 -> > C: Server connected
21:00:27.773 ->
21:00:27.773 -> Check the capability...
21:00:27.826 -> > C: Check the capability
21:00:27.926 ->
21:00:27.926 -> Send command, STARTTLS
21:00:27.926 -> > C: Send STARTTLS command
21:00:27.973 -> > C: Perform the SSL/TLS handshake
21:00:27.973 -> > C: seeding the random number generator
21:00:27.973 -> > C: setting up the SSL/TLS structure
21:00:27.973 -> ! W: Skipping SSL Verification. INSECURE!
21:00:27.973 -> > C: setting hostname for TLS session
21:00:27.973 -> > C: perform the SSL/TLS handshake
21:00:29.929 -> > C: verifying peer X.509 certificate
21:00:29.929 ->
21:00:29.929 -> Check the capability...
21:00:29.929 -> > C: Check the capability
21:00:30.377 ->
21:00:30.377 -> Logging in...
21:00:30.377 -> > C: Send IMAP command, LOGIN
21:00:30.878 ->
21:00:30.878 -> Check the capability...
21:00:30.878 -> > C: Check the capability
21:00:30.931 -> Successfully logged in.
21:00:30.931 ->
21:00:30.931 -> Reading the list of mailboxes...
21:00:30.931 -> > C: Send IMAP command, LIST
21:00:31.231 ->
21:00:31.231 -> Available folders: INBOX, &g0l6P3ux-, &XfJT0ZAB-, &XfJSIJZk-, &V4NXPpCuTvY-, &Xn9USpCuTvY-, &i6KWBZCuTvY-, Notes, Archive, &j4Nl6XaEkK5O9lkHTv0-
21:00:31.278 ->
21:00:31.278 -> Selecting the INBOX folder...
21:00:31.278 -> > C: Open the mailbox folder
21:00:31.426 -> Error, EXAMINE Unsafe Login. Please contact [email protected] for help
21:00:31.426 -> > E: EXAMINE Unsafe Login. Please contact [email protected] for help

<!-- LEAVE THIS LINE AS-IS AND DON'T DELETE IT, OTHERWISE THE ISSUE WILL BE CLOSED AUTOMATICALLY. -->

[mobizt]

Ok I will add support this in the next update.

[dingxiao88]

Thanks, I am looking forward to it very much.

[mobizt]

Now the library updated to v3.1.0. The IMAP ID is now supported by default.

ESP-Mail-Client/examples/IMAP/Read_Single_Email/Read_Single_Email.ino

Lines 246 to 255 in 51c1070

	// If ID extension was supported by IMAP server, assign the client identification
	// name, version, vendor, os, os_version, support_url, address, command, arguments, environment
	// Server ID can be optained from imap.serverID() after calling imap.connect and imap.id.
	// imap_data.identification.name = "User";
	// imap_data.identification.version = "1.0";
	/* Connect to the server */
	if (!imap.connect(&config, &imap_data))
	return;

If sever support ID and no imap_data.identification keys assigned, the default ID will be sent instead.

[dingxiao88]

Thank you very much for your hard work. I have successfully verified and can now receive emails normally. Thanks again.