codehaus


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

The speed of glob()


On 29/07/2020 11.43, Gisle Vanem wrote:
> Chris Angelico wrote:
> 
>>> Has anybody noticed the speed of 'glob()' has
>>> decreased somewhere between v3.6 and v3.10.
>>>
>>> I got these results:
>>> ??? Python 3.6.5:
>>> ????? 1st run: 0.14694
>>> ????? 2nd run: 0.09506?? <- *always* the fastest
>>> ??? Python 3.7.7:??????? <- from Nuget
>>> ????? 1st run: 0.12440
>>> ????? 2nd run: 0.09602
>>> ??? Python 3.10.0:?????? <- from Git repo
>>> ????? 1st run: 0.15922
>>> ????? 2nd run: 0.12424
>>>
>>> 'glob()' in Python 3.6.5 is consistently 30% faster on
>>> the 2nd run compared to 3.10.0.
>>>
>>
>> Unfortunately that's very hard to compare. If you're building Python
>> from source, the parameters may be VERY different from a prepackaged
>> binary.
> 
> I guess so w/o knowing why. I just tried this from MS' AppStore:
> Python 3.8.5:
> ? 1st run: 0.12121
> ? 2nd run: 0.07674
> 
> Fastest I've tried so far.

Official builds and most distribution builds are optimized PGO builds
(PGO = profile guided optimization). It's a multi-step build approach.
At first the build system creates a special instrumented version of
Python that collects runtime information. Then it executes tests to
collect information how the code is used. Finally the build system
builds Python again. The compiler uses the runtime information to create
optimized code.

Some distributions also use special compiler flags or other tricks to
make Python even faster. Victor Stinner (a Python core dev) and his
co-workers have done some research to speed up Python by up to 30% on
top of PGO,
https://developers.redhat.com/blog/2020/06/25/red-hat-enterprise-linux-8-2-brings-faster-python-3-8-run-speeds/

Christian