gh-96512: Modernize int_max_str_digits protection in 3.12

Doc/c-api/init_config.rst

@@ -828,6 +828,18 @@ PyConfig
 
  Default: ``0``.
 
+ .. c:member:: int32_t int_max_str_digits
+
+ If greater than 0, enable int conversion digit limitations. ``-1`` means
+ that :data:`sys.int_info.default_max_str_digits` will be used.
+
+ Configured by the :option:`-X int_max_str_digits <-X>` command line
+ flag or the :envvar:`PYTHONINTMAXSTRDIGITS` environment varable.
+
+ Default: ``-1``.
+
+ .. versionadded:: 3.12
+
  .. c:member:: int isolated
 
  If greater than ``0``, enable isolated mode:

Doc/library/functions.rst

@@ -910,7 +910,7 @@ are always available. They are listed here in alphabetical order.
  .. versionchanged:: 3.11
  The delegation to :meth:`__trunc__` is deprecated.
 
- .. versionchanged:: 3.12
+ .. versionchanged:: 3.11
  :class:`int` string inputs and string representations can be limited to
  help avoid denial of service attacks. A :exc:`ValueError` is raised when
  the limit is exceeded while converting a string *x* to an :class:`int` or

Doc/library/json.rst

@@ -258,7 +258,7 @@ Basic Usage
  be used to use another datatype or parser for JSON integers
  (e.g. :class:`float`).
 
- .. versionchanged:: 3.12
+ .. versionchanged:: 3.11
  The default *parse_int* of :func:`int` now limits the maximum length of
  the integer string via the interpreter's :ref:`integer string
  conversion length limitation <int_max_str_digits>` to help avoid denial

Doc/library/stdtypes.rst

@@ -5523,6 +5523,22 @@ Verification:
  ... '571186405732').to_bytes(53, 'big')
  ...
 
+.. versionadded:: 3.11
+
+.. audit-event:: int/digits/to_decimal value
+
+ When a conversion from an :class:`int` to decimal is estimated to be larger
+ than :data:`sys.int_info.str_digits_check_threshold <sys.int_info>` digits
+ an :ref:`auditing event <auditing>` with the :class:`int` ``value`` as the
+ argument is raised.
+
+.. audit-event:: int/digits/from_base digits base
+
+ When a conversion to an :class:`int` from a base other than 2, 4, 8, 16, or
+ 32 has more than :data:`sys.int_info.str_digits_check_threshold
+ <sys.int_info>` digits an :ref:`auditing event <auditing>` with the number
+ of ``digits`` and ``base`` is raised.
+
 .. versionadded:: 3.12
 
 Affected APIs
@@ -5578,7 +5594,7 @@ Information about the default and minimum can be found in :attr:`sys.int_info`:
 * :data:`sys.int_info.str_digits_check_threshold <sys.int_info>` is the lowest
  accepted value for the limit (other than 0 which disables it).
 
-.. versionadded:: 3.12
+.. versionadded:: 3.11
 
 .. caution::
 

Doc/library/sys.rst

@@ -544,7 +544,7 @@ always available.
  .. versionchanged:: 3.11
  Added the ``safe_path`` attribute for :option:`-P` option.
 
- .. versionchanged:: 3.12
+ .. versionchanged:: 3.11
  Added the ``int_max_str_digits`` attribute.
 
 
@@ -732,7 +732,7 @@ always available.
  Returns the current value for the :ref:`integer string conversion length
  limitation <int_max_str_digits>`. See also :func:`set_int_max_str_digits`.
 
- .. versionadded:: 3.12
+ .. versionadded:: 3.11
 
 .. function:: getrefcount(object)
 
@@ -1029,7 +1029,7 @@ always available.
 
  .. versionadded:: 3.1
 
- .. versionchanged:: 3.12
+ .. versionchanged:: 3.11
  Added ``default_max_str_digits`` and ``str_digits_check_threshold``.
 
 
@@ -1337,7 +1337,7 @@ always available.
  <int_max_str_digits>` used by this interpreter. See also
  :func:`get_int_max_str_digits`.
 
- .. versionadded:: 3.12
+ .. versionadded:: 3.11
 
 .. function:: setprofile(profilefunc)
 

Doc/library/test.rst

@@ -1018,7 +1018,7 @@ The :mod:`test.support` module defines the following functions:
  context to allow execution of test code that needs a different limit
  on the number of digits when converting between an integer and string.
 
- .. versionadded:: 3.12
+ .. versionadded:: 3.11
 
 
 The :mod:`test.support` module defines the following classes:

Doc/using/cmdline.rst

@@ -585,7 +585,7 @@ Miscellaneous options
  .. versionadded:: 3.11
  The ``-X frozen_modules`` option.
 
- .. versionadded:: 3.12
+ .. versionadded:: 3.11
  The ``-X int_max_str_digits`` option.
 
  .. versionadded:: 3.12
@@ -775,7 +775,7 @@ conflict.
  interpreter's global :ref:`integer string conversion length limitation
  <int_max_str_digits>`.
 
- .. versionadded:: 3.12
+ .. versionadded:: 3.11
 
 .. envvar:: PYTHONIOENCODING
 

Doc/whatsnew/3.12.rst

@@ -84,15 +84,11 @@ Other Language Changes
  (Contributed by Serhiy Storchaka in :gh:`87995`.)
 
 * Converting between :class:`int` and :class:`str` in bases other than 2
- (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)
- now raises a :exc:`ValueError` if the number of digits in string form is
- above a limit to avoid potential denial of service attacks due to the
- algorithmic complexity. This is a mitigation for `CVE-2020-10735
- <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10735>`_.
- This limit can be configured or disabled by environment variable, command
- line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion
- length limitation <int_max_str_digits>` documentation. The default limit
- is 4300 digits in string form.
+ (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) now
+ generates an :ref:`auditing event <auditing>` whenever the estimated number
+ of digits is above :data:`sys.int_info.str_digits_check_threshold
+ <sys.int_info>`. See the :ref:`integer string conversion length limitation
+ <int_max_str_digits>` documentation.
 
 * :class:`memoryview` now supports the half-float type (the "e" format code).
  (Contributed by Dong-hee Na and Antoine Pitrou in :gh:`90751`.)

Include/cpython/initconfig.h

@@ -178,6 +178,7 @@ typedef struct PyConfig {
  wchar_t *check_hash_pycs_mode;
  int use_frozen_modules;
  int safe_path;
+ int32_t int_max_str_digits;
 
  /* --- Path configuration inputs ------------ */
  int pathconfig_warnings;

Include/internal/pycore_initconfig.h

@@ -79,6 +79,9 @@ PyAPI_FUNC(PyStatus) _PyArgv_AsWstrList(const _PyArgv *args,
 PyAPI_FUNC(int) _Py_str_to_int(
  const char *str,
  int *result);
+PyAPI_FUNC(int) _Py_str_to_int32(
+ const char *str,
+ int32_t *result);
 PyAPI_FUNC(const wchar_t*) _Py_get_xoption(
  const PyWideStringList *xoptions,
  const wchar_t *name);
@@ -170,8 +173,6 @@ extern void _Py_DumpPathConfig(PyThreadState *tstate);
 
 PyAPI_FUNC(PyObject*) _Py_Get_Getpath_CodeObject(void);
 
-extern int _Py_global_config_int_max_str_digits; // TODO(gpshead): move this into PyConfig in 3.12 after the backports ship.
-
 
 /* --- Function used for testing ---------------------------------- */
 

Include/internal/pycore_interp.h

@@ -175,7 +175,7 @@ struct _is {
  struct types_state types;
  struct callable_cache callable_cache;
 
-int int_max_str_digits;
+int32_t int_max_str_digits;
 
  /* The following fields are here to avoid allocation during init.
  The data is exposed through PyInterpreterState pointer fields.

Lib/test/audit-tests.py

@@ -7,6 +7,7 @@
 
 import contextlib
 import os
+from test import support
 import sys
 
 
@@ -409,8 +410,6 @@ def hook(event, *args):
 
 
 def test_sys_getframe():
- import sys
-
  def hook(event, args):
  if event.startswith("sys."):
  print(event, args[0].f_code.co_name)
@@ -430,10 +429,24 @@ def hook(event, args):
  _wmi.exec_query("SELECT * FROM Win32_OperatingSystem")
 
 
-if __name__ == "__main__":
- from test.support import suppress_msvcrt_asserts
+def test_int_digits():
+ def hook(event, args):
+ if event.startswith("int/digits/"):
+ print(event, *(hex(arg) for arg in args))
+
+ sys.addaudithook(hook)
+
+ threshold = sys.int_info.str_digits_check_threshold
+ with support.adjust_int_max_str_digits(2*threshold):
+ fives = int('5' * (threshold+20))
+ int('1' * (threshold*3//2), base=36)
+ for base in (2, 4, 8, 16, 32): # linear unaudited bases
+ int('1' * (threshold+1), base)
+ str(fives)
 
- suppress_msvcrt_asserts()
+
+if __name__ == "__main__":
+ support.suppress_msvcrt_asserts()
 
  test = sys.argv[1]
  globals()[test]()

Lib/test/test_audit.py

@@ -200,5 +200,32 @@ def test_wmi_exec_query(self):
  self.assertEqual(actual, expected)
 
 
+ def test_int_digits(self):
+ returncode, events, stderr = self.run_python("test_int_digits")
+ if returncode:
+ self.fail(stderr)
+
+ if support.verbose:
+ print(*events, sep='\n')
+ actual = [(ev[0], ev[2].split(' ')) for ev in events]
+
+ threshold = sys.int_info.str_digits_check_threshold
+ self.assertEqual(
+ actual.pop(0),
+ ("int/digits/from_base", [hex(threshold+20), hex(10)]),
+ )
+ self.assertEqual(
+ actual.pop(0),
+ ("int/digits/from_base", [hex(threshold*3//2), hex(36)]),
+ )
+
+ event = actual.pop(0)
+ self.assertEqual(event[0], "int/digits/to_decimal")
+ self.assertEqual(len(event[1]), 1)
+ self.assertGreater(len(event[1][0]), threshold//2)
+
+ self.assertEqual(actual, [])
+
+
 if __name__ == "__main__":
  unittest.main()

Lib/test/test_embed.py

@@ -434,6 +434,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
  'install_signal_handlers': 1,
  'use_hash_seed': 0,
  'hash_seed': 0,
+ 'int_max_str_digits': -1,
  'faulthandler': 0,
  'tracemalloc': 0,
  'perf_profiling': 0,

Misc/NEWS.d/next/Core and Builtins/2022-09-15-16-11-38.gh-issue-96512.VrlORK.rst

@@ -0,0 +1,4 @@
+Added ``int/digits/to_decimal`` and ``int/digits/from_base`` :ref:`auditing
+events <auditing>` on :class:`int` non-linear algorithm base conversions
+with more than `sys.int_info.str_digits_check_threshold <sys.int_info>`
+digits.

Objects/longobject.c

@@ -1766,8 +1766,11 @@ long_to_decimal_string_internal(PyObject *aa,
  */
  if (size_a >= 10 * _PY_LONG_MAX_STR_DIGITS_THRESHOLD
  / (3 * PyLong_SHIFT) + 2) {
+ if (PySys_Audit("int/digits/to_decimal", "O", aa) < 0) {
+ return -1;
+ }
  PyInterpreterState *interp = _PyInterpreterState_GET();
-int max_str_digits = interp->int_max_str_digits;
+int32_t max_str_digits = interp->int_max_str_digits;
  if ((max_str_digits > 0) &&
  (max_str_digits / (3 * PyLong_SHIFT) <= (size_a - 11) / 10)) {
  PyErr_Format(PyExc_ValueError, _MAX_STR_DIGITS_ERROR_FMT_TO_STR,
@@ -1837,7 +1840,7 @@ long_to_decimal_string_internal(PyObject *aa,
  }
  if (strlen > _PY_LONG_MAX_STR_DIGITS_THRESHOLD) {
  PyInterpreterState *interp = _PyInterpreterState_GET();
-int max_str_digits = interp->int_max_str_digits;
+int32_t max_str_digits = interp->int_max_str_digits;
  Py_ssize_t strlen_nosign = strlen - negative;
  if ((max_str_digits > 0) && (strlen_nosign > max_str_digits)) {
  Py_DECREF(scratch);
@@ -2513,8 +2516,11 @@ digit beyond the first.
 
  /* Limit the size to avoid excessive computation attacks. */
  if (digits > _PY_LONG_MAX_STR_DIGITS_THRESHOLD) {
+ if (PySys_Audit("int/digits/from_base", "ni", digits, base) < 0) {
+ return NULL;
+ }
  PyInterpreterState *interp = _PyInterpreterState_GET();
-int max_str_digits = interp->int_max_str_digits;
+int32_t max_str_digits = interp->int_max_str_digits;
  if ((max_str_digits > 0) && (digits > max_str_digits)) {
  PyErr_Format(PyExc_ValueError, _MAX_STR_DIGITS_ERROR_FMT_TO_INT,
  max_str_digits, digits);
@@ -6196,7 +6202,7 @@ _PyLong_InitTypes(PyInterpreterState *interp)
  return _PyStatus_ERR("can't init int info type");
  }
  }
- interp->int_max_str_digits = _Py_global_config_int_max_str_digits;
+ interp->int_max_str_digits = _PyInterpreterState_GetConfig(interp)->int_max_str_digits;
  if (interp->int_max_str_digits == -1) {
  interp->int_max_str_digits = _PY_LONG_DEFAULT_MAX_STR_DIGITS;
  }