Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigInt/Int64 negative value parse error #15200

Closed
2 tasks done
GaussETH opened this issue Jan 24, 2025 · 3 comments · Fixed by #15230
Closed
2 tasks done

BigInt/Int64 negative value parse error #15200

GaussETH opened this issue Jan 24, 2025 · 3 comments · Fixed by #15230
Labels
confirmed-bug We've confirmed this is a bug in Mongoose and will fix it.
Milestone

Comments

@GaussETH
Copy link

Prerequisites

  • I have written a descriptive issue title
  • I have searched existing issues to ensure the bug has not already been reported

Mongoose version

8.9.5

Node.js version

22

MongoDB server version

8.0.4

Typescript version (if applicable)

No response

Description

BigInt field with negative value parse error in latest mongoose(8.9.5), bug works well in 7.8.4.

Steps to Reproduce

Image

Model

const UserRewardSchema = new Schema({
  chainId: {
    type: Number,
    required: true,
    immutable: true,
  },
  account: {
    type: String,
    required: true,
    index: true,
    immutable: true,
  },
  value: {
    type: BigInt,
    required: true,
    default: 0n,
  },
}, {
  versionKey: false,
  timestamps: true,
  collection: 'userRewards',
})
UserRewardSchema.index({ chainId: 1, account: 1 }, { unique: true })

export const UserReward = model('UserReward', UserRewardSchema)

Image

import { connect } from "mongoose"
import { DB_URI } from "#common/env.js"

/**
 * @type {import('mongoose').Mongoose}
 */
let mongoose

export const connectDb = async () => {
  if (!mongoose) {
    mongoose = await connect(DB_URI, { useBigInt64: true })
    mongoose.set("transactionAsyncLocalStorage", true).set('strictQuery', true)
  }

  return mongoose
}
const items = await UserReward.find({ account: 'xxxxx' })
    .select('-_id')
    .lean()

console.log(items)

Expected Behavior

Image

@vkarpov15 vkarpov15 added this to the 8.9.6 milestone Jan 24, 2025
@vkarpov15 vkarpov15 added the has repro script There is a repro script, the Mongoose devs need to confirm that it reproduces the issue label Jan 24, 2025
@sudhackar
Copy link

Simpler repro

diff --git a/test/bigint.test.js b/test/bigint.test.js
index e3d00418e..14e3bddac 100644
--- a/test/bigint.test.js
+++ b/test/bigint.test.js
@@ -88,7 +88,7 @@ describe('BigInt', function() {
     let Test;

     before(async function() {
-      db = await start();
+      db = await start({useBigInt64: true});

       const schema = new Schema({
         myBigInt: BigInt
@@ -114,14 +114,14 @@ describe('BigInt', function() {
     });

     it('becomes a bigint with lean using useBigInt64', async function() {
-      await Test.create({ myBigInt: 7n });
+      await Test.create({ myBigInt: 18446744073709551613n });

       const doc = await Test.
-        findOne({ myBigInt: 7n }).
+        findOne({ myBigInt: 18446744073709551613n }).
         setOptions({ useBigInt64: true }).
         lean();
       assert.ok(doc);
-      assert.strictEqual(doc.myBigInt, 7n);
+      assert.strictEqual(doc.myBigInt, -3n);
     });

     it('can query with comparison operators', async function() {

This test fails on current version - but I think its not due to code changes in mongoose

@sudhackar
Copy link

git bisect points to 750c6e2

this will pass the test mentioned above

diff --git a/package.json b/package.json
index 663a81830..990478ec3 100644
--- a/package.json
+++ b/package.json
@@ -19,9 +19,9 @@
   ],
   "license": "MIT",
   "dependencies": {
-    "bson": "^6.10.1",
+    "bson": "5.5.0",
     "kareem": "2.6.3",
-    "mongodb": "~6.12.0",
+    "mongodb": "5.7.0",
     "mpath": "0.9.0",
     "mquery": "5.0.0",
     "ms": "2.1.3",

@vkarpov15 vkarpov15 modified the milestones: 8.9.6, 8.9.7 Jan 30, 2025
@vkarpov15
Copy link
Collaborator

This issue seems to be indicative of not setting useBigInt64. For example, the following script stores the correct value normally, but stores -3n if useBigInt64 option is commented out. Did you always have await connect(DB_URI, { useBigInt64: true }) or did you recently add useBigInt64?

'use strict';

const mongoose = require('mongoose');
mongoose.set('debug', true);

(async function() {
  await mongoose.connect('mongodb://127.0.0.1:27017/mongoose_test', { useBigInt64: true });

  const schema = new mongoose.Schema({
    myBigInt: BigInt
  });
  const Test = mongoose.model('Test', schema);
  await Test.deleteMany({});

  const { _id } = await Test.create({ myBigInt: 18446744073709551613n });
  const doc = await Test.findById(_id);
  doc.markModified('myBigInt');
  await doc.save();

  console.log(await Test.findById(_id));
})();

This does bring up a good point that useBigInt64 with save() is a bit of a footgun. Without useBigInt64, BigInts can lose precision when updating documents in place. We will work on fix.

@vkarpov15 vkarpov15 added confirmed-bug We've confirmed this is a bug in Mongoose and will fix it. and removed has repro script There is a repro script, the Mongoose devs need to confirm that it reproduces the issue labels Feb 3, 2025
vkarpov15 added a commit that referenced this issue Feb 3, 2025
@vkarpov15 vkarpov15 modified the milestones: 8.9.7, 8.11 Feb 3, 2025
vkarpov15 added a commit that referenced this issue Feb 6, 2025
fix(bigint): throw error when casting BigInt that's outside of the bounds of what MongoDB can safely store
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
confirmed-bug We've confirmed this is a bug in Mongoose and will fix it.
Projects
None yet
3 participants