The Python 3 Wall of Shame

Here’s my attempt at motivating package maintainers to port to python 3 and you can check out the code that generates the chart. I basically scraped PyPI (13,000 webpages) for this info and formatted it into an HTML table which is uploaded to appspot. I added a ‘cron’ task on my PC to recrawl PyPI every Sunday so the chart stays fresh. When developing the WOS I used filecache (from the previous blog post) so I could just write the code that parses the crawl as though it just now scraped PyPI while infact everything was cached to disk. Without filecache I would have had to either wait for ages, or write code that stores the scrapes, and reparses them. About 10 lines turn into an import and a decorator, now that’s magic.

I hope no one takes offense from this. The situation we’re starting at is pretty bad. Only 11 out of the top 100 packages on PyPI are labeled as supporting python3. There are a few glitches (e.g. multiprocessing) where the developers simply haven’t yet labeled the package as python 3 compliant.

Once we pass the 50% mark I guess we can change it from “Wall of Shame” to “World of Strength” or something because the subdomain is “wos”.

31 thoughts on “The Python 3 Wall of Shame

  1. First off, nice website! Looks good and easily communicates what it needs to.

    But there are a couple of things you could do to improve the reporting. One, It isn’t so much that e.g., multiprocessing have not updated their labels, it’s that they never will thanks to the modules being included in the stdlib. I count four projects which have been merged into Python 3.2 (or earlier) and thus are implicitly ported to Python 3.

    You also count Django 3 times: 1.2.3, 1.2.1, and 1.2.4. You might want to only consider the latest release instead of (in Django’s case) triple counting it as Django is not about to add support for Python 3 in a micro release.

    Lastly, you may want to color projects that specify *no* specific Python version in another way (e.g., docutils). If they are not going to bother to specify even what Python 2 versions they rely on then they are most certainly not going to bother to list Python 3 support. And in docutils’ case it does have Python 3 support.

    But thanks for doing this! It’s nice to visibly see this sort of thing.

  2. unittest2 has python 3 support through a different distribution (same package name if you install it): unittest2py3k. There are python 3 version of virtualenv and docutils too.

    multiprocessing and uuid are both in the standard library now and so aren’t *needed* as separate python 3 versions.

    • I just now hardcoded ‘multiprocessing’, ‘simplejson’, ‘argparse’, and ‘uuid’ as irrelevant to the WOS. So I can manually add ‘unittest2’, ‘virtualenv’ and ‘docutils’. Though I would have preferred an automatic way of figuring this out so I don’t have to manually remove each package that forks…

      • Yeah, I agree hardcoding is nasty. Unless pypi provides me with metadata to describe the fact that unittest2 for python 3 is a different package I don’t see how to do it though. It would certainly be incorrect to say that the unittest2 distribution itself works with python 3 (it doesn’t).

        As it happens I’m *considering* switching to a single distribution (shipping both source code packages in a single distribution and deciding at install time which to install – this isn’t without its own problems however).

  3. A couple notes:
    – distribute is the py3k compatible version of setuptools, so setuptools shouldn’t be listed at all (it will never turn green)
    – pymongo was ported to py3k by yours truly (it’s named pymongo3 on pypi) and it might take a very long time before the package under the “pymongo” name will be py3k compatible

    • As noted, we need a way to link from py2 packages to their py3 counterparts or to devise a way of containing both packages in one We should ask python-dev about this.

      • Neither method will fix the setuptools showing up as red issue. The author of setuptools has essentially refused to acknowledge distribute as the legitimate successor to setuptools, despite the project being led by the distutils maintainer. You may still have to hardcode this case.

  4. pyOpenSSL supports Python 3, but you have it listed as red. How useful is the website if so much of the data it presents is wrong?

    • If you’ll add the python 3 classifier to pyOpenSSL then you’ll get the right color. I’m sorry for the inconvenience. Is there another way I could have extrapolated pyOpenSSL supports py3k? If there is then I don’t mind implementing it.

      • Likewise, is there a way to indicate that a distribution is the py3k counterpart for another? See my previous comment about two distributions where this would be necessary.

  5. numpy has python 3 support from long ago. There is a python 3.1 package in pypi. However in your list it is indicated in red.

  6. What about packages that change name when they go to Python3? e.g., virtualenv is Python 2 only, but the virtualenv5 fork is Python 2 and 3 compatible. PyPI makes it a total pain in the ass to release Py2 and Py3 libraries under the same name but with different source packages. e.g. my twitter package is Python2 and twitter3 is Python3.

    • For some distributions, like setuptools, this is an appropriate concern. For virtualenv, however, it is not because virtualenv is actively developed and could get official Python 3 support as early as next week (during the PyCon sprints).
      As for your twitter package, could you not use two different source directories for py2 and py3 respectively and choose which one to use in your based on the Python version used? I recall that there are some distributions using this packaging style already (although I can’t name any).

  7. How is this for wall of shame…PyMongo will not install with Python 3.2 installed. The installer insists that Python 3.1 is required.

    I found this website while googling with the following search string:
    “pymongo3 does not install with python 3.2”

    • The Windows installers for Python 3.2 were missing, that’s all. Those installers have now been uploaded to PyPI.

  8. Pingback: Python 3 Library updates, via twitter « Code and comments

  9. IPython does have a Python3.x branch. I use IPython0.11 with Python3.2 everyday, and it works really well. You can use it online at PythonAnywhere dot something. They should be made green.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s