Skip to content

Add guppy backend - #256

Open
dxe4 wants to merge 2 commits into
pythonprofilers:masterfrom
dxe4:add_guppy_backend
Open

Add guppy backend#256
dxe4 wants to merge 2 commits into
pythonprofilers:masterfrom
dxe4:add_guppy_backend

Conversation

@dxe4

@dxe4dxe4 commented Nov 2, 2019

Copy link
Copy Markdown

related to my comment here #236 (comment)
with implementation of guppy3 we can get the right values.

note this implementation does self.prevlines[-1] == self.prev_lineno: for guppy backend only, because repeating [i for i in range(0, 10000000)] makes the guppy backend extremely slow. although this may not be ideal or acceptable i thought it's worth sharing the code.


@profile
def myfn_py():
ones = [1] * NBR_ITEMS
twos = [i + 1 for i in ones]
return twos
@profile
def myfn_np():
ones_np = np.ones(NBR_ITEMS)
twos_np = ones_np+1
return twos_np
myfn_np()
myfn_py()
python -m memory_profiler --backend guppy prof.py Line # Mem usage Increment Line Contents
================================================
11 14.191 MiB 14.191 MiB @profile
12 def myfn_py():
13 18.006 MiB 3.815 MiB ones = [1] * NBR_ITEMS
14 22.098 MiB 4.091 MiB twos = [i + 1 for i in ones]
15 return twos
Filename: prof.py
Line # Mem usage Increment Line Contents
================================================
18 14.189 MiB 14.189 MiB @profile
19 def myfn_np():
20 18.005 MiB 3.817 MiB ones_np = np.ones(NBR_ITEMS)
21 21.820 MiB 3.815 MiB twos_np = ones_np+1
22 21.820 MiB 0.000 MiB return twos_np

Comment threadmemory_profiler.py
if self.backend == 'guppy':
if self.prevlines[-1] == self.prev_lineno:
# the reason for this is because for code like [i for i in range(0, 1000000)] guppy will be extremely slow
_run = False

@dxe4dxe4Nov 2, 2019

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can cause issues if you want to measure the memory of a side effect eg

[foo() for i in range(0, 500)]
def foo():
make_big_array = [i for i in random.randint(0, int(1e9))]

i haven't tested it.
although it will give correct results for this case #236 (comment)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@dxe4