SIGBUS: writing to `mmap`ed device beyond file size

[Semnodime]

Crash report

What happened?

# DEV=/dev/foo
# blockdev --getsize64 $DEV
1048576
# tail $DEV
# python3.12 bug.py
Bus error (core dumped)
# tail $DEV
hell#

from mmap import mmap

with open('/dev/foo', 'r+b') as file:
 m = mmap(file.fileno(), 1024 ** 2 + 1)

m.seek(1024 ** 2 - 4)

for b in b'hello world!':
 m.write(bytes([b]))

I hope you don't mind my reference to Madagascar

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.12.3 (main, Apr 27 2024, 19:00:21) [GCC 11.4.0]

[Zheaoli]

May I ask if the /sda3 is your disk device？

[Zheaoli]

Can not reproduced on Linux 6.9

[Semnodime]

@Zheaoli The bug is reproducible on python3.12.3 hosted on Linux kernel 6.9.2-1-MANJARO.

I suppose you overlooked the blockdev --getsize64 $DEV statement which implies the exact blockdevice size.

[manjaro@manjaro ~]$ sudo sh
sh-5.2# uname -a
Linux manjaro 6.9.2-1-MANJARO #1 SMP PREEMPT_DYNAMIC Mon May 27 03:56:18 UTC 2024 x86_64 GNU/Linux
sh-5.2# python -VV
Python 3.12.3 (main, Apr 23 2024, 09:16:07) [GCC 13.2.1 20240417]
sh-5.2# modprobe brd rd_nr=1 rd_size=1024
sh-5.2# DEV=/dev/ram0
sh-5.2# SIZE=$(blockdev --getsize64 $DEV)
sh-5.2# tail $DEV
sh-5.2# python3.12 -c "from mmap import mmap;file=open('$DEV','r+b');m=mmap(file.fileno(),$SIZE+1);m.seek($SIZE-4);[m.write(bytes([b])) for b in b'hello world!']"
Bus error (core dumped)
sh-5.2# tail $DEV
hellsh-5.2# 

Notice the hellsh (hell shell) at the end.

[Zheaoli]

The issue confirmed, But I think it's not Python bug I think

The core reason here is that the file size is 1048576 but you map a 1048577 memory to it. So the process SIGBUS now

You need to take care of the SIGBUS when you use the mmap API

[eryksun]

On POSIX, mmap operations can raise SIGBUS or SIGSEV, depending on the platform. Nothing is currently implemented to handle either signal. On Windows, mmap handles the corresponding exceptions for EXCEPTION_IN_PAGE_ERROR and EXCEPTION_ACCESS_VIOLATION by raising OSError instead of letting the default exception handler terminate the process.

[Semnodime]

@eryksun
Is there unreasonable overhead if cpython is updated to also raise an OSError on POSIX?

[duaneg]

The Windows code uses structured exception handling to do that, which is very different from POSIX signal handlers. It will complex to do this with signal handlers, if it is possible at all. Perhaps using setjmp/longjmp with thread-local buffers, but that would impose significant overhead on what might be a very hot code path.

It would also be a change in behaviour to install a handler for SIGBUS, not to mention SIGSEV(!), when we previously didn't.

Personally, as a POSIX mmap enjoyer, a SIGBUS/SIGSEV is exactly what I would expect and want to get in this scenario. YMMV of course.

[serhiy-storchaka]

This issue can also be reproduced with mapping to a regular file and then truncating it. Except on Windows, where you cannot simply truncate a mapped file.

import mmap
start_size = 2 * mmap.PAGESIZE
reduced_size = mmap.PAGESIZE
f = open('/tmp/gh119817', 'wb+')
f.truncate(start_size)
m = mmap.mmap(f.fileno(), start_size)
f.truncate(reduced_size)
m[reduced_size]

Actually, #66234 is caused by using mmap() internally in gdbm.