Planet Python

Last update: May 18, 2017 11:23 AM

May 18, 2017

Python Anywhere

System update this morning

This morning's system update went well :-)

There aren't any major visible new features in the new system -- it was primarily an infrastructural change. The operating system on our underlying servers has been upgraded from Ubuntu 14.04 to 16.04 (so we had to rewrite all of our upstart system jobs as systemd ones, which was... fun). The sandboxes where your code runs have been kept as Ubuntu 14.04, so that your code doesn't break due to the system it runs on changing unexpectedly.

A future update, hopefully soon, will enable a 16.04-based system image that you'll be able to opt in to use when it's convenient to you. There's also a big new feature that we're working on that required the OS upgrade -- more about that another time...

There were a few minor changes beyond that:

We've increased the server capacity available for scheduled tasks.
We've significantly reworked the system for logging of error messages for web apps, so they should appear pretty much instantly in your error logs rather than being delayed -- previously they could sometimes take 30 seconds to arrive, which could make debugging frustrating.
We've been working on improving the API; there's some documentation here. The API is still in early alpha, and may change without warning, but if you'd like to have a play, just let us know and we can switch it on for your account.
A couple of security fixes -- mostly rate-limiting things that send email so that they can't be abused by trolls trying to fill up your inbox with junk. A shout-out to akash c for reporting that.

May 18, 2017 07:36 AM

Daniel Bader

When to Use Python

What is the Python programming language used for in the real world, and when is using Python the right choice?

When I grew up in Germany as a kid there was this craze about “desks that can grow with you.” The idea was you’d buy your kid an adjustable desk and then they’d be able to use it throughout their whole education career.

As your kid grows taller, so does his or her desk. Just turn the little crank handle every few months… And voila, you’re right on track for raising the next Albert Einstein or Marie Curie.

Python is a great
“adjustable desk” language.

With the small but important difference that Python is also a much prettier desk. One that you wouldn’t be embarrassed of using past elementary school. And one you’d be okay with showing to your girlfriend/boyfriend. (Okay, time to stop with that desk analogy.)

My point is this:

What I love about Python is how it scales so well (no pun intended)—from writing simple prototypes to validate an idea, all the way to building “production grade” systems.

Sure, sometimes it would be nice to have a compiler and static type checks to lean on—but often I realized that I would’ve never come this far in so little time with Java or C++. And with optional type hints in Python 3 and type checking tools like mypy this gap is starting to close.

But not only does Python scale and grow with the project at hand, it also scales and grows with your skills as a developer.

It’s relatively easy to get started with Python—but it’s not going to prevent you from growing as a developer and getting impressive real-world work done with it. My friend and fellow Python wrangler Michael Kennedy refers to it as a “full spectrum” language. And I really like that as an analogy.

Python spans the gamut from print('hello, world') all the way to running the back-end infrastructure for massive applications like Reddit, Instagram, or YouTube.

Now, is using Python
always the right choice?

No.

No single programming language is.

For example, it’s unlikely you’re going to write a real-time operating system kernel in Python. Neither will id Software use it to implement their next-generation rendering engine…

But millions of developers around the world are using Python to build web applications, write data-crunching pipelines, generate reports, automate tests, conduct research, and do all kinds of other amazing work in a multitude of domains.

By learning Python you’re not limiting yourself to a specific niche.

And that’s what I love about this adorable, “adjustable desk” of a language.

Happy Pythoning!

May 18, 2017 12:00 AM

May 17, 2017

Sandipan Dey

Some more Computational Photography: Creating Video Textures in Python and OpenCV

The following problem appeared as an assignment in the Coursera Course Computational Photography (by Georgia Tech, 2013). The following description of the problem is taken directly from the assignment’s description. ntroduction In this article, we shall be applying our computational photography magics to video, with the purpose of creating video textures, or infinitely looping pieces … Continue reading Some more Computational Photography: Creating Video Textures in Python and OpenCV

May 17, 2017 09:57 PM

Andre Roberge

What if range did not exist?

Over the years, various proposals for new syntactic constructs have been put forward to supplement or replace the range() function for looping over integers. Some of them have been documented in PEPs, whereas various others arose during discussions on the python-dev, comp.lang.python, and python-ideas lists. This lead Guido van Rossum to write in 2005: The whole point (15 years ago) of

May 17, 2017 05:03 PM

Python Engineering at Microsoft

Come visit us at PyCon US 2017!

With PyCon US 2017 happening later this week, we wanted to let you know what we will be up to at the conference!

To start, thanks to our platinum sponsorship this year, we are going to have a booth in the exhibit hall. This will provide you a place to come if you have any questions involving Python and Microsoft. We are also going to have a constant rotation of 10 and 20 minute demos at our booth that will include:

Come by our booth to see the schedule for when we will be demoing what!

But our participation doesn’t stop at sponsorship and demos. Both Steve Dower and myself will be attending the Python language summit. This is the once-a-year meeting where the core developers of Python have an opportunity to get together in a room and discuss various points and issues surrounding the language. We will also be attending the development sprints where we sprint with other core developers to resolve various design decisions, build some camaraderie, and generally just get some coding done!

Lastly, both Steve and I are presenting at the conference this year. Steve is giving his talk entitled
“I Installed Python 3.6 on Windows and I Liked It” where he will discuss how Python 3.6 makes cross-platform support easier for Windows and UNIX easier. I will be giving my talk entitled “What’s new in Python 3.6” where I will cover the over 16 new features in Python 3.6 that make it one of Python’s largest release ever.

As with every year we expect to have a great time at PyCon US and we look forward to see those of you who are able to make it!

May 17, 2017 05:00 PM

Weekly Python Chat

PyCon Sprints Chat

We'll chat at the sprints at PyCon 2017!

May 17, 2017 05:00 PM

PyCon Chat

Join us live from Pycon 2017 in Portland, Oregon!

May 17, 2017 05:00 PM

NumFOCUS

What is it like to chair a PyData conference?

Have you ever wondered what it’s like to be in charge of a PyData event? Vincent Warmerdam has collected some thoughts and reflections on his experience chairing this year’s PyData Amsterdam conference: This year I was the chair of PyData Amsterdam and I’d like to share some insights on what that was like. I was on the committee the […]

May 17, 2017 03:09 PM

Experienced Django

Hello, Heroku!

This week I finally deployed the KidTasks app to Heroku. While I was very happy with Heroku and how it behaved, I’ll have to admit that many of the shortcuts I’ve taken in developing this first app have come back to bite me during this process. None of the issues are things I can blame Heroku for. They were all self-inflicted.

Help!

To get started, I walked through the demo application instructions on Heroku and deployed the demo app they have. I thought this demo was really well put together and explained things quite well. Unfortunately, there were several requirements that I hadn’t yet met. Fortunately, most of those were easily solved.

I had only been running the app locally and using the built-in Django server. This is great for what it is, but not recommended for an actual deployment.

It’s Easy

The first several changes were fairly trivial. They recommend using gunicorn; I’ll use gunicorn. pip install; pip freeze; git add requirements.txt and that’s all set.

Similarly they recommend dj_database_url for reading the database settings out of the environment. Also easily installed and the code modifications they showed worked without issue.

The White Album

The next suggested package was WhiteNoise which manages serving static content from Django. I honestly had never thought of this before and had some learning to do. The install was fine, but I ran into issues with the bootstrap fonts not being present during the collectstatic operation. It took me a bit of digging but I figured out a solution for this – I put the bootstrap fonts in the static/bootstrap/fonts directory. I’m not sure if this is the preferred solution, but it works for now and keeps all the files local.

At this point I attempted my first deploy. Of course, it failed hard.

But the logs were kind enough to point out the problem:

CommandError: You must set settings.ALLOWED_HOSTS if DEBUG is False.

and, doing some searching on the web and the example app, the answer was easy, simply set the field to * in settings.py

# Allow all host headers
ALLOWED_HOSTS = ['*']

It was also at this point that it occurred to me that I had to make other changes to my settings.

Do You Want To Know A Secret?

Setting the debug flag to False was an obvious change in settings, but what to do about that pesky secret key? Heroku (and the Two Scoops book) both recommend using an environment variable, but several sites on the web point out that environment variables are less secure than a file on the filesystem. While this may be true in general (“cat /proc//environ” on a linux box will give you the environment variables), it doesn’t appear to be on Heroku’s dynos.

Another idea was to use a dynamic secret key for the app. This can be done with code (from SO) that mirrors how Django initially creates these keys:

from django.utils.crypto import get_random_string

chars = 'abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)'
get_random_string(50, chars)

While this would work for this instance, there are some problems with it and it seems like using “best practices” as general rule would be a good idea, so I followed the directions and set it in the environment.

Mr. Post(gres)Man

Next came the big challenge. I had been using sqlite3 for my app because, well, it’s easy. Heroku doesn’t support sqlite3 as explained in the first answer in this SO post.

So, I needed to switch to a new database. As postgres was the recommended solution and I didn’t have another preference, I went with that.

This conversion actually took a good portion of the entire process, again, largely through self-inflicted problems.

The basic setup for running postgres on Linux I gleaned from this site.

The dj_database_url and heroku’s default database config settings were fine. There were just a couple of things I needed to learn first. I needed to set up the postgres database with a full user, grant rights, etc. Doing this locally, it took me a while to figure out all the the steps. But, once I had done this, transferring that to Heroku was very straightforward. You can to

heroku run bash

to get into the dyno, or for this case

heroku pg:psql

to get you directly into the postgres prompt on the production database.

Get Back

The final hurdle turned out to be with the database. I had forgotten that you need to specify the app name when doing makemigrations and migrate. I kept getting errors starting the app because it couldn’t find any of the required tables for the app. It turns out that

heroku run python manage.py makemigrations
heroku run python manage.py migrate

acted like they succeeded but didn’t create the required tables. After a bit of late-night head scratching, it finally dawned on me to add the name of the app

heroku run python manage.py makemigrations tasks
heroku run python manage.py migrate tasks

Once I got that solved the app was up and running.

Full Speed Ahead, Mr. Boatswain!

Mainly because I will likely find it handy in the future, I’m going to list some of the commands I found useful in the adventure.

heroku local web – test the configuration on your local system
heroku create kidtasks – create a new heroku app with a name
git push heroku master– this triggers a full deploy of the code
heroku config:set SECRET_KEY=<my super secret key>– sets environment variables on the app’s dyno
heroku run python manage.py [makemigrations tasks | etc] – basic django commands on the deployed app
heroku logs –tail – get the logs from the app

The End

While this process took more effort than I expected, I don’t think Heroku is to blame for any of it. Now that I have a working example (and an idea of how postgres works), the next time will be much smoother.

Next app I start, I will definitely be using this template and see how much time it saves me.

May 17, 2017 01:07 PM

Mike Driscoll

Educative Python Courses on Sale for PyCon!

I am putting my interactive Educative Python courses on Sale for PyCon this week. You can get Python 101 and Python 201: Intermediate Python for 50% off. Here are the coupon codes you can use:

For Python 101 – au-pycon-py101
For Python 201 – au-pycon-py201

Educative is also doing a 50% off sale on their Python 3: An interactive deep dive course which you can get with this coupon: au-pycon-deepdive.

And now for something completely different, Educative is offering a 17% off sale of their Coderust 2.0: Faster Coding Interview Preparation using Interactive Visualizations, so if you are interested in learning something a little different, now is your chance! Here’s the code for that: au-pycon-coderust

May 17, 2017 12:30 PM

EuroPython

EuroPython 2017: First list of accepted sessions available

We have received an amazing collection of proposals. Thank you all for your submissions! Given the overwhelming quality of the proposals, we had some very difficult decisions to make. Nonetheless we are happy to announce we have published the first 140+ sessions.

Here’s what we have on offer so far:

for a total of 138 sessions in addition to the 3 keynotes we have already announced.

More sessions to come

More sessions will be announced early next week and the full schedule will follow. In total, we will again have more than 200 sessions waiting for you.

Please see the session list for details and abstracts. In case you wonder what poster, interactive and help desk sessions are, please check the call for proposals.

Aside: If you haven’t done yet, please get your EuroPython 2017 ticket soon. We will switch to on-desk rates in June, which will cost around 30% more than the regular rates.

Enjoy,
–
EuroPython 2017 Team
EuroPython Society
EuroPython 2017 Conference

May 17, 2017 10:10 AM

Python Piedmont Triad User Group

PYPTUG Monthly meeting "Seeking Exotics" May 30th

Come join PYPTUG at out next monthly meeting (May 30th 2017) to learn more about the Python programming language, modules and tools. Python is the ideal language to learn if you've never programmed before, and at the other end, it is also the perfect tool that no expert would do without. Monthly meetings are in addition to our project nights.

What

Meeting will start at 6:00pm.

We will open on an Intro to PYPTUG and on how to get started with Python, PYPTUG activities and members projects, then on to News from the community.

Main Talk: "Seeking Exotics"

by Francois Dion, Data Scientist, Inmar Inc.

Abstract:

Medicare payments, UPC code descriptions, fertility rate and fires. All of it is data, some of which is erroneous and some of which is anomalous. Seeking Exotics introduces the audience to the world of outliers and anomaly detection through the use of metrics, visualizations and open source machine learning tools.

Lightning talks!

We will have some time for extemporaneous "lightning talks" of 5-10 minute duration. If you'd like to do one, some suggestions of talks were provided here, if you are looking for inspiration. Or talk about a project you are working on.

When

Tuesday, May 30th 2017Meeting starts at 6:00PM

Where

Wake Forest University, close to Polo Rd and University Parkway:
Manchester Hall

room: Manchester 241

Wake Forest University, Winston-Salem, NC 27109

Map this

And speaking of parking: Parking after 5pm is on a first-come, first-serve basis. The official parking policy is:

"Visitors can park in any general parking lot on campus. Visitors should avoid reserved spaces, faculty/staff lots, fire lanes or other restricted area on campus. Frequent visitors should contact Parking and Transportation to register for a parking permit."

Mailing List

Don't forget to sign up to our user group mailing list:

https://groups.google.com/d/forum/pyptug?hl=en

It is the only step required to become a PYPTUG member.

RSVP on meetup:

meetup.com/PYthon-Piedmont-Triad-User-Group-PYPTUG/events/238568416/

May 17, 2017 04:34 AM

May 16, 2017

Mike Driscoll

wxPython: Learning about TreeCtrls

The wxPython GUI toolkit comes with many widgets. A common control is a tree widget. wxPython has several different tree widgets, including the regular wx.TreeCtrl, the newer DVC_TreeCtrl and the pure Python variants, CustomTreeCtrl and HyperTreeList. In this article, we will focus on the regular wx.TreeCtrl and learn the basics of how to create and use one.

Creating a Simple Tree

Creating a TreeCtrl is actually quite easy. The wxPython demo has a fairly complex example, so I wasn’t able to use it here. Instead I ended up taking the demo example and stripping it down as much as I could. Here’s the result:

import wx
 
class MyTree(wx.TreeCtrl):
 
    def __init__(self, parent, id, pos, size, style):
        wx.TreeCtrl.__init__(self, parent, id, pos, size, style)
 
 
class TreePanel(wx.Panel):
 
    def __init__(self, parent):
        wx.Panel.__init__(self, parent)
 
        self.tree = MyTree(self, wx.ID_ANY, wx.DefaultPosition, wx.DefaultSize,
                           wx.TR_HAS_BUTTONS)    
 
        self.root = self.tree.AddRoot('Something goes here')
        self.tree.SetPyData(self.root, ('key', 'value'))
        os = self.tree.AppendItem(self.root, 'Operating Systems')
        self.tree.Expand(self.root)
 
        sizer = wx.BoxSizer(wx.VERTICAL)
        sizer.Add(self.tree, 0, wx.EXPAND)
        self.SetSizer(sizer)
 
 
class MainFrame(wx.Frame):
 
    def __init__(self):
        wx.Frame.__init__(self, parent=None, title='TreeCtrl Demo')
        panel = TreePanel(self)
        self.Show()
 
 
if __name__ == '__main__':
    app = wx.App(redirect=False)
    frame = MainFrame()
    app.MainLoop()

In this example, we create a subclass of wx.TreeCtrl that doesn’t do anything. Then we create a panel subclass where we instantiate the tree and add a root and sub-item. Finally we create the frame that holds the panel and run the application. You should end up with something that looks similar to the following:

This is a pretty boring example, so let’s make something a bit more interesting.

Creating an XML Viewer

Something that I’ve wanted to do for some time now is to create an XML editor with Python. To get started, I wrote some code a couple of weekends ago that could read XML into a TreeCtrl for viewing the tag elements. For this example, I will be using some sample XML I found on Microsoft’s MSDN website:

<?xml version="1.0"?>
<catalog>
    <book id="bk101">
        <author>Gambardella, Matthew</author>
        <title>XML Developer's Guide</title>
        <genre>Computer</genre>
        <price>44.95</price>
        <publish_date>2000-10-01</publish_date>
        <description>An in-depth look at creating applications 
        with XML.</description>
    </book>
    <book id="bk102">
        <author>Ralls, Kim</author>
        <title>Midnight Rain</title>
        <genre>Fantasy</genre>
        <price>5.95</price>
        <publish_date>2000-12-16</publish_date>
        <description>A former architect battles corporate zombies, 
        an evil sorceress, and her own childhood to become queen 
        of the world.</description>
    </book>
    <book id="bk103">
        <author>Corets, Eva</author>
        <title>Maeve Ascendant</title>
        <genre>Fantasy</genre>
        <price>5.95</price>
        <publish_date>2000-11-17</publish_date>
        <description>After the collapse of a nanotechnology 
        society in England, the young survivors lay the 
        foundation for a new society.</description>
    </book>
    <book id="bk104">
        <author>Corets, Eva</author>
        <title>Oberon's Legacy</title>
        <genre>Fantasy</genre>
        <price>5.95</price>
        <publish_date>2001-03-10</publish_date>
        <description>In post-apocalypse England, the mysterious 
        agent known only as Oberon helps to create a new life 
        for the inhabitants of London. Sequel to Maeve 
        Ascendant.</description>
    </book>
    <book id="bk105">
        <author>Corets, Eva</author>
        <title>The Sundered Grail</title>
        <genre>Fantasy</genre>
        <price>5.95</price>
        <publish_date>2001-09-10</publish_date>
        <description>The two daughters of Maeve, half-sisters, 
        battle one another for control of England. Sequel to 
        Oberon's Legacy.</description>
    </book>
    <book id="bk106">
        <author>Randall, Cynthia</author>
        <title>Lover Birds</title>
        <genre>Romance</genre>
        <price>4.95</price>
        <publish_date>2000-09-02</publish_date>
        <description>When Carla meets Paul at an ornithology 
        conference, tempers fly as feathers get ruffled.</description>
    </book>
    <book id="bk107">
        <author>Thurman, Paula</author>
        <title>Splish Splash</title>
        <genre>Romance</genre>
        <price>4.95</price>
        <publish_date>2000-11-02</publish_date>
        <description>A deep sea diver finds true love twenty 
        thousand leagues beneath the sea.</description>
    </book>
    <book id="bk108">
        <author>Knorr, Stefan</author>
        <title>Creepy Crawlies</title>
        <genre>Horror</genre>
        <price>4.95</price>
        <publish_date>2000-12-06</publish_date>
        <description>An anthology of horror stories about roaches,
        centipedes, scorpions  and other insects.</description>
    </book>
    <book id="bk109">
        <author>Kress, Peter</author>
        <title>Paradox Lost</title>
        <genre>Science Fiction</genre>
        <price>6.95</price>
        <publish_date>2000-11-02</publish_date>
        <description>After an inadvertant trip through a Heisenberg
        Uncertainty Device, James Salway discovers the problems 
        of being quantum.</description>
    </book>
    <book id="bk110">
        <author>O'Brien, Tim</author>
        <title>Microsoft .NET: The Programming Bible</title>
        <genre>Computer</genre>
        <price>36.95</price>
        <publish_date>2000-12-09</publish_date>
        <description>Microsoft's .NET initiative is explored in 
        detail in this deep programmer's reference.</description>
    </book>
    <book id="bk111">
        <author>O'Brien, Tim</author>
        <title>MSXML3: A Comprehensive Guide</title>
        <genre>Computer</genre>
        <price>36.95</price>
        <publish_date>2000-12-01</publish_date>
        <description>The Microsoft MSXML3 parser is covered in 
        detail, with attention to XML DOM interfaces, XSLT processing, 
        SAX and more.</description>
    </book>
    <book id="bk112">
        <author>Galos, Mike</author>
        <title>Visual Studio 7: A Comprehensive Guide</title>
        <genre>Computer</genre>
        <price>49.95</price>
        <publish_date>2001-04-16</publish_date>
        <description>Microsoft Visual Studio 7 is explored in depth,
        looking at how Visual Basic, Visual C++, C#, and ASP+ are 
        integrated into a comprehensive development 
        environment.</description>
    </book>
</catalog>

The first thing we need to decide is what Python XML parser we want to use. I personally like lxml the best, but Python’s own ElementTree is certainly a viable option and actually quite easy to switch too if you start out with lxml. But for this example, we will be using lxml. Let’s take a look:

import wx
 
from lxml import etree, objectify
 
 
class XmlTree(wx.TreeCtrl):
 
    def __init__(self, parent, id, pos, size, style):
        wx.TreeCtrl.__init__(self, parent, id, pos, size, style)
 
        try:
            with open(parent.xml_path) as f:
                xml = f.read()
        except IOError:
            print('Bad file')
            return
        except Exception as e:
            print('Really bad error')
            print(e)
            return
 
        self.xml_root = objectify.fromstring(xml)
 
        root = self.AddRoot(self.xml_root.tag)
        self.SetPyData(root, ('key', 'value'))        
 
        for top_level_item in self.xml_root.getchildren():
            child = self.AppendItem(root, top_level_item.tag)
            self.SetItemHasChildren(child)
            if top_level_item.attrib:
                self.SetPyData(child, top_level_item.attrib)
 
        self.Expand(root)
        self.Bind(wx.EVT_TREE_ITEM_EXPANDING, self.onItemExpanding)
 
    def onItemExpanding(self, event):
        item = event.GetItem()
        book_id = self.GetPyData(item)
 
        for top_level_item in self.xml_root.getchildren():
            if top_level_item.attrib == book_id:
                book = top_level_item
                self.SetPyData(item, top_level_item)                                
                self.add_book_elements(item, book)
                break
 
    def add_book_elements(self, item, book):
        for element in book.getchildren():
            child = self.AppendItem(item, element.tag)
            if element.getchildren():
                self.SetItemHasChildren(child)
 
            if element.attrib:
                self.SetPyData(child, element.attrib)  
 
 
class TreePanel(wx.Panel):
 
    def __init__(self, parent, xml_path):
        wx.Panel.__init__(self, parent)
        self.xml_path = xml_path
 
        self.tree = XmlTree(self, wx.ID_ANY, wx.DefaultPosition, wx.DefaultSize,
                            wx.TR_HAS_BUTTONS)    
 
        sizer = wx.BoxSizer(wx.VERTICAL)
        sizer.Add(self.tree, 0, wx.EXPAND)
        self.SetSizer(sizer)
 
 
class MainFrame(wx.Frame):
 
    def __init__(self, xml_path):
        wx.Frame.__init__(self, parent=None, title='XML Editor')
        panel = TreePanel(self, xml_path)
        self.Show()
 
 
if __name__ == '__main__':
    xml_path = 'books.xml'
    app = wx.App(redirect=False)
    frame = MainFrame(xml_path)
    app.MainLoop()

The main change here is in the TreeCtrl subclass, although we had to make some small modifications in the other classes to pass in the XML file path. Let’s focus on the TreeCtrl class though. First we read the XML from the file and load it into lxml’s objectify module. At this point, we have an XML object that we can use to populate the TreeCtrl with data. So we add the root and then loop over the top-level children in the XML. For each top level element, we add an item to the root of the TreeCtrl. This is extremely basic as we should also be checking each element to see if it has children too. We don’t. Instead we just assume that it does and call the TreeCtrl’s SetItemHasChildren() method. This adds an arrow to the element to allow expanding of the element.

Lastly we expand the root and bind an event to EVT_TREE_ITEM_EXPANDING which will allow us to update the sub-elements when they get expanded. You can see how this is done in the onItemExpanding event handler and the add_book_elements() which is called by the event handler. Here we actually DO check the element to see if it has children using lxml’s getchildren(). If it does, then we call SetItemHasChildren(). The other thing I want to point out is all the calls to SetPyData(). The SetPyData() method is for saving data into the tree item. In this case, we are saving the XML element into the tree item itself, which we can get access to again via GetPyData(). This will be important if we want to add editing functionality to the GUI.

Other Odds and Ends

The wxPython demo also demonstrates some interesting tidbits. For example, it shows that you can add a wx.ImageList to your TreeCtrl. It also shows some of the other tree specific events you can bind to. Such as:

EVT_TREE_ITEM_COLLAPSED
EVT_TREE_SEL_CHANGED
EVT_TREE_BEGIN_LABEL_EDIT
EVT_TREE_END_LABEL_EDIT
EVT_TREE_ITEM_ACTIVATED

Of course, you can also bind to mouse events like EVT_LEFT_DCLICK and EVT_RIGHT_DOWN too.

If you’d like to make the tree elements editable, then you’ll need to pass the wx.TR_EDIT_LABELS style flag in when you instantiate the tree. Since my example is just a viewer, I didn’t feel the need to do that. There are some other style flags mentioned in the demo and in the documentation that you may also want to check out.

Wrapping Up

At this point, I think you should be able to get started using wxPython’s handy wx.TreeCtrl. It is quite powerful and easy to utilize. Should you find yourself needing to do something more custom, then I highly recommend checking out one of wxPython’s alternate tree controls, such as the CustomTreeCtrl or HyperTreeList.

PyBites

How to Parse Common Data Formats in Python

In this post we demonstrate ways in which you can parse common data formats used in Python.

May 16, 2017 01:41 PM

S. Lott

Needless Complexity -- Creating Havoc Leads to Mistakes

I received the worst code example ever. The. Worst.

Here's the email.

I have hurriedly created a blog post titled [omitted] at the url below
[also omitted]
...
Unfortunately, I am neither an algorithm expert or a Python expert. However, I am willing to jump in.

Please review the Python code snippets. I know that they work because I ran them using Python 2.7.6. It was the environment available on my work PC. Speed to get something to the group so that it does not disband outweighs spending time on environments. The goal is not to be Pythonic but to have anyone that has written any code follow the logic.

Also, please review the logic. Somehow, I managed to get the wrong answer. The entire blog post is a build to provide a solution to CLRS exercise 2.3-7. My analysis gave me the answer of O( {n [log n]}**2 ) and the CLRS answer is O(n [log n] ). Where did I screw up my logic?

CLRS is https://mitpress.mit.edu/books/introduction-algorithms.

The referenced blog post is shocking. The "neither an algorithm expert or a Python expert" is an understatement. The "willing to jump in" is perhaps a bad thing. I sent several comments. They were all ignored. I asked for changes a second time. That was also ignored. Eventually, changes were made reluctantly and only after a distressing amount of back-and-forth.

Havoc was created through a process of casually misstating just about everything that can possibly be misstated. It transcended mere "wrong" and enters that space where the whole thing can't even be falsified. It was beyond simply wrong.

My point here is (partially) to heap ridicule on the author. More importantly, want to isolate a number of issues to show how simple things can become needlessly complex and create havoc.

The Problem

The definition of the problem 2.3-7 seems so straightforward.

"Describe a $\Theta(n \log n)$-time algorithm that, given a set $S$ of $n$ integers and another integer $x$, determines whether or not there exist two elements in $S$ whose sum is exactly $x$."

This brings us to problem #1. The blog post is unable to actually repeat the problem. Here's the quote:

x + y = x0
where
x ∈ N and y ∈ N for some finite set of integer values N
x0: the integer to which x and y sum
x != y

It's not at all clear what's going on here. What's x0? What's N? Why is x!=y even introduced?

This is how havoc starts. The requirements have been restated in a way that makes them more confusing. The original terminology was dropped in favor of random new terminology. There's no reason for restating the problem. The consequence of the bad restatement is to introduce needless features and create confusion.

The Python Nonsense

A quote:

Concepts are demonstrated via code snippets. They code snippets were executed using Python 2.7.6. They were written in such a way that anyone with basic coding skills could read the code. In other words, the goal was not to be Pythonic.

Python 2.7.6 has been obsolete since May of 2014. At the very least, use a current release.

The goal of using Python without being Pythonic seems to be -- well -- schizophrenic. Or it's intentional troll-bait. Hard to say.

Another paragraph says this.

The Python community will be annoyed because I am using Python 2 and not 3. Their annoyance is appropriate. Unfortunately, I only have Windows machines and can't afford to screw them up at this point in time.

What? That makes no sense at all. It's trivial to install Python 3.6 side-by-side with Python 2. Everyone should. Right now. I'll wait. See https://conda.io/docs/. Start here: https://conda.io/miniconda.html.

If you're going to insist on using the quirky and slow Python 2, you absolutely must use this in all of your code:

from __future__ import print_function, division, absolute_import, unicode_literals

Python 2 code without this is wrong. If you're still using Python 2, add this to all your code, right now. Please. You'll have to fix stuff that breaks; but we'll all thank you for it. pylint --py3k will help you locate and fix this.

The code with a -2/10 pylint score

I'm trying to reproduce this faithfully. It's hard, because the original blog post has issues with layout.

SomeIntegerList = [1, 2, 3, 4, 5, 6]
DesiredSumOfIntegers = 11
for SomeIntegerA in SomeIntegerList:
 for SomeIntegerB in SomeIntegerList:
  if SomeIntegerA == SomeIntegerB: continue
        SumOfIntegers = SomeIntegerA + SomeIntegerB
        print "SomeInteger A = ", SomeIntegerA, ", SomeInteger B = ", SomeIntegerB, ", Sum of Integers = ", SumOfIntegers
  if DesiredSumOfIntegers == SumOfIntegers:
      print "DesiredSumOfIntegers = ", DesiredSumOfIntegers, " was found"

(The original really could not be copied and pasted to create code that could even be parsed. I may have accidentally fixed that. I hope not.)

Almost every line of code has a problem. It gets worse, of course.

There's output in the original blog post that provides a hint as to what's supposed to be happening here.

Addition is Commutative

Yes. There is an entire paragraph plus a spreadsheet which proves that addition is commutative. An. Entire. Paragraph. Plus. A. Spreadsheet.

Meanwhile, factorial, multiplication, and division aren't mentioned. Why do we need a spreadsheet to show that addition is commutative, yet, all other operators are ignored? No clue. Moving on.

Permutations

A quote:

Now, let's talk about the number of computations involved in using nested for loops to examine all the possible addition permutations. Here I am using the term permutation as it is strictly defined in mathematics.

First. The algorithm uses all combinations. $\textbf{O}(n^2)$.

Second. "as it is strictly defined in mathematics" should go without saying. If you feel the need to say this, it calls the entire blog post into question.

It's like "honestly." Anyone who has to establish their honesty with "can I be honest with you?" is still lying.

If we're being strict here, are we not being strict elsewhere? If we're not being strict, why not?

The algorithm enumerates all combinations of n things taken 2 at a time without replacement. For reasons that aren't clear. The original problem statement permits replacement. The restatement of the problem doesn't permit replacement.

The n things taken r or 2 at a time problem

There's a table with values for $\frac{n!}{(n-r)!}$

No hint is given as to what this table is or why it's here. I think it's supposed to be because of this:

$\frac{n!}{r!(n-r)!} \text{ with } r=2 \equiv \frac{n!}{2(n-2)!} \equiv \frac{n\times(n-1)}{2}$

It's hard to say why commutativity of addition gets a paragraph, but this gets no explanation at all. To me, it shows a disregard for the reader: the reader doesn't understand addition, but they totally get factorial.

Another Perspective

A quote

Another perspective is to note that the nested for loops result in O(n^2). Clearly, the above approach is not scalable.

That's not "another perspective." That's. The. Point. The entire point of the exercise is that the brute force algorithm isn't optimal.

The Worst Code Snippet Ever

This is truly and deeply shocking.

SomeIntegerList = [1, 2, 3, 4, 5, 6]
DesiredSumOfIntegers = 11
i = 0
for SomeIntegerA in SomeIntegerList:
    i = i + 1
    j = 0
 for SomeIntegerB in SomeIntegerList:
        j = j + 1
        if j > i:
            print "i = ", i, ", j = ", j
            SumOfIntegers = SomeIntegerA + SomeIntegerB
            print "SomeInteger A = ", SomeIntegerA, ", SomeInteger B = ", SomeIntegerB, ", Sum of Integers = ", SumOfIntegers
      if DesiredSumOfIntegers == SumOfIntegers:
          print "DesiredSumOfIntegers = ", DesiredSumOfIntegers, " was found"

This is what drove me over the edge. This is unconscionably evil programming. It transcends mere "non-Pythonic" and reaches a realm of hellish havoc that can barely be understood as rational. Seriously. This is evil incarnate.

This is the most baffling complex version of a half-matrix iteration that I think I've ever seen. I can only guess that this is written by someone uncomfortable with thinking. They copied and pasted a block of assembler code changing the syntax to Python. I can't discern any way to arrive at this code.

The Big-O Problem

This quote:

Even though the number of computations is cut in half

The rules for Big-O are in the cited CLRS book. $\textbf{O}(\frac{n^2}{2}) = \textbf{O}(n^2)$.

The "cut in half" doesn't count when describing the overall worst-case complexity. It needs to be emphasized that "cut in half" doesn't matter. Over and over again.

This code doesn't solve the problem. It doesn't advance toward solving the problem. And it's unreadable. Maybe it's a counter-example? An elaborate "don't do this"?

The idea of for i in range(len(S)): for j in range(i): ... seems to be an inescapable approach to processing the upper half of a matrix, and it seems to be obviously $\textbf{O}(n^2)$.

The Binary Search

This quote is perhaps the only thing in the entire blog post that's not utterly wrong.

we can compute the integer value that we need to find. We can than do a search over an ordered list for the integer that we need to find.

Finally. Something sensible. Followed by more really bad code.

The code starts with this

def binarySearch(alist, item):

instead of this

from bisect import bisect

Why does anyone try to write code when Python already provides it?

There's more code, but it's just badly formatted and has a net pylint score that's below zero. We've seen enough.

There's some further analysis that doesn't make any sense at all:

Since the integers that sum must be distinct, the diagnol on the matrix have values of N/A

And this:

Secondly, we should remove the integer that we are on from the binary search

This is a consequence of the initial confusion that decided that $x \neq y$ was somehow part of the problem. When it wasn't. These two sentences indicate a level of profound confusion about the essential requirements. Which leads to havoc.

Added Complication

The whole story is pretty badly confused. Then this arrives.

Complicate Problem by Having Integer List Not Sorted

It's not clear what this is or why it's here. But there it is.

It leads eventually to this, which also happens to be true.

The total computation complexity is O(2 * n [log n] ) = O(n [log n] )

That's not bad. However. The email that asked for help claimed O( {n [log n]}**2 ). I have no idea what the email is talking about. Nor could I find out what any of this meant.

The Kicker

The kicker is some code that solves the problem in $\textbf{O}(n)$ time. Without using a set, which is interesting.

This was not part of the CLRS exercise 2.3-7. I suppose it's just there to point out something about something. Maybe it's a "other people are smarter than CLRS"? Or maybe it's a "just google for the right answer without too much thinking"? Hard to say.

A sentence or two of introduction might be all that's required to see why the other result is there.

Lessons Learned

Some people like to add complexity to the problem. The $x \neq y$ business is fabricated from thin air. It adds to the code complexity, but is clearly not part of the problem space.

This creates havoc. Simple havoc.

Some people appear to act like they're asking for help. But they're not. They may only want affirmation. A nice pat on the head. "Yes, you've written a blog post." Actual criticism isn't expected or desired. This is easy to detect by the volume and vehemence of the replies.

Given a list of numbers, S, and a target, x, determine of two values exist in the set that sum to x.

>>> S = [1,2,3,4,5,6]
>>> x=11
>>> [(n, x-n) for n in S if (x-n) in S]
[(5, 6), (6, 5)]
>>> bool([(n, x-n) for n in S if (x-n) in S])
True

This follows directly from the analysis. It doesn't add anything new or different. It just uses Python code rather than indented assembler.

This first example is $\textbf{O}(n^2)$ because the in operator is applied to a list. We can, however, use bisect() instead of the in operator.

>>> [(n, x-n) for n in S if S[bisect(S, (x-n))-1] == x-n]
[(5, 6), (6, 5)]
>>> x=13
>>> [(n, x-n) for n in S if S[bisect(S, (x-n))-1] == x-n]
[]

This achieves the goal -- following the parts of the analysis that aren't riddled with errors -- without so much nonsensical code.

This does require some explanation for what bisect(S, i) does. It's important to note that the bisect() function returns the position at which we should insert a new value to maintain order. It doesn't return the location of a found item. Indeed, if the item isn't found, it will still return a position into which a new item should be inserted.

If we want this to be $\textbf{O}(n)$, we can use this:

>>> S = [1,2,3,4,5,6]
>>> S_set = set(S)
>>> x=11
>>> bool([(n, x-n) for n in S_set if (x-n) in S_set])
True

This replaces the linear list with a set, S_set. The (x-n) in S_set operation is $\textbf{O}(1)$, leading to the overall operation being $\textbf{O}(n)$.

If you want to shave a little time, you can use any() instead of bool([]). If you're not returning the pairs, you can reduce it to any(x-n in S_set for n in S_set). Try it with timeit to see what the impact is. It's surprisingly small.

May 16, 2017 08:00 AM

Wingware Blog

Using Wing Pro for Remote Python Web Development

Wing Pro is a Python IDE that can be used to develop, test, and debug a website on a remote server, VM, or other system where an IDE cannot be installed.

May 16, 2017 01:00 AM

Daniel Bader

Stacks in Python

How to implement a stack data structure (LIFO) in Python using built-in types and classes from the standard library.

A stack is a collection of objects that supports fast last-in, first-out (LIFO) semantics for inserts and deletes. Unlike lists or arrays, stacks typically don’t allow for random access to the objects they contain. The insert and delete operations are also often called push and pop.

A useful real-world analogy for a stack data structure is a stack of plates:

New plates are added to the top of the stack. And because the plates are precious and heavy, only the topmost plate can be moved (last-in, first-out). To reach the plates lower down in the stack the topmost plates must be removed one by one.

Stacks and queues are similar. They’re both linear collections of items and the difference lies in the order that items are accessed in:

With a queue you remove the item least recently added (first-in, first-out or FIFO); and with a stack you remove the item most recently added (last-in, first-out or LIFO).

Performance-wise, a proper stack implementation is expected to take O(1) time for insert and delete operations.

Stacks have a wide range of uses in algorithms, for example in language parsing and runtime memory management (“call stack”). A short and beautiful algorithm using a stack is depth-first search (DFS) on a tree or graph data structure.

Python ships with several stack implementations that each have slightly different characteristics. Let’s take a look at them:

✅ The list Built-in

Python’s built-in list type makes a decent stack data structure as it supports push and pop operations in amortized O(1) time.

Python’s lists are implemented as dynamic arrays internally which means they occasional need to resize the storage space for elements stored in them when elements are added or removed. The list over-allocates its backing storage so that not every push or pop requires resizing and you get an amortized O(1) time complexity for these operations.

The downside is that this makes their performance less consistent than the stable O(1) inserts and deletes provided by a linked list based implementation (like collections.deque, see below). On the other hand lists do provide fast O(1) time random access to elements on the stack which can be an added benefit.

Here’s an important performance caveat when using lists as stacks:

To get the amortized O(1) performance for inserts and deletes new items must be added to the end of the list with the append() method and removed again from the end using pop(). Stacks based on Python lists grow to the right and shrink to the left.

Adding and removing from the front is much slower and takes O(n) time, as the existing elements must be shifted around to make room for the new element.

# How to use a Python list as a stack (LIFO):

s = []

s.append('eat')
s.append('sleep')
s.append('code')

>>> s
['eat', 'sleep', 'code']

>>> s.pop()
'code'
>>> s.pop()
'sleep'
>>> s.pop()
'eat'

>>> s.pop()
IndexError: "pop from empty list"

✅ The collections.deque Class

The deque class implements a double-ended queue that supports adding and removing elements from either end in O(1) time (non-amortized).

Because deques support adding and removing elements from either end equally well, they can serve both as queues and as stacks.

Python’s deque objects are implemented as doubly-linked lists which gives them excellent and consistent performance for inserting and deleting elements, but poor O(n) performance for randomly accessing elements in the middle of the stack.

collections.deque is a great choice if you’re looking for a stack data structure in Python’s standard library with the performance characteristics of a linked-list implementation.

# How to use collections.deque as a stack (LIFO):

from collections import deque
q = deque()

q.append('eat')
q.append('sleep')
q.append('code')

>>> q
deque(['eat', 'sleep', 'code'])

>>> q.pop()
'code'
>>> q.pop()
'sleep'
>>> q.pop()
'eat'

>>> q.pop()
IndexError: "pop from an empty deque"

✅ The queue.LifoQueue Class

This stack implementation in the Python standard library is synchronized and provides locking semantics to support multiple concurrent producers and consumers.

The queue module contains several other classes implementing multi-producer, multi-consumer queues that are useful for parallel computing.

Depending on your use case the locking semantics might be helpful, or just incur unneeded overhead. In this case you’d be better off with using a list or a deque as a general purpose stack.

# How to use queue.LifoQueue as a stack:

from queue import LifoQueue
s = LifoQueue()

s.put('eat')
s.put('sleep')
s.put('code')

>>> s
<queue.LifoQueue object at 0x108298dd8>

>>> s.get()
'code'
>>> s.get()
'sleep'
>>> s.get()
'eat'

>>> s.get_nowait()
queue.Empty

>>> s.get()
# Blocks / waits forever...

A good default choice: `collections.deque`

If you’re not looking for parallel processing support (or don’t want to handle locking and unlocking manually) your choice comes down to the built-in list type or collections.deque.

The difference lies in the data structure used behind the scenes and ease of use.

list is backed by a dynamic array which makes it great for fast random access but requires occasional resizing when elements are added or removed. The list over-allocates its backing storage so that not every push or pop requires resizing and you get an amortized O(1) time complexity for these operations. But you do need to be careful to only insert and remove items from the right-hand side (append and pop) or otherwise performance slows down to O(n).
collections.deque is backed by a doubly-linked list which optimizes appends and deletes at both ends and provides consistent O(1) performance for these operations. Not only is its performance more stable, the deque class is also easier to use because you don’t have to worry about adding or removing items from “the wrong end.”

For these reasons, collections.deque makes an excellent choice for implementing a stack (LIFO queue) data structure in Python.

Read the full “Fundamental Data Structures in Python” article series here. This article is missing something or you found an error? Help a brother out and leave a comment below.

May 16, 2017 12:00 AM

Thomas Guest

Lazy sequences working hard

I gave a talk @PyDiff this evening in the computer science department at Cardiff University.

Lazy Sequences working hard

Python has no problem handling large and even infinite streams of data. Just write lazy programs — code which defers data access until the last minute. This talk examines Python’s language and library support for such delaying tactics. There will be live coding, and we’ll draw parallels with similar features in other languages, in particular the Unix shell.

Being unsure where to pitch it, I started off easy and kept going until I’d lost everybody — including myself.

The room was well set up with a good quality projector and whiteboard, along with a desk to sit down when I wanted to write and run code and plenty of space to move around in otherwise. I did feel a bit like a jack-in-the-box by the end.

I’d based the talk on a Jupyter notebook which I replayed with the ingenious RISE reveal.js extension written by Damian Avila. This worked well, since I got the pretty graphics along with the interactive coding. A static version of the slides is available here.

Thanks to everyone who came. Sorry I had to rush off after. If anyone would like to talk at Swansea, please let me know: you’d be most welcome.

May 16, 2017 12:00 AM

May 15, 2017

Python 4 Kids

3: Getting Help

Book ref: Project 1 (pg 24ff)

Python 2.7: Same

PyCon

Don’t Overlook the Open Spaces at PyCon this Year

[A guest post by PyCon 2017’s Open Spaces Chair, Anna Ossowski!]

Open Spaces are one of the most often overlooked activities at the PyCon conference.

PyCon is not merely a 5-track conference — it’s true there are 5 tracks of talks, but there are also 5 tracks of Open Spaces that run alongside the talks.

What are Open Spaces?

Open Spaces are self-organizing meetup-like events which occur in parallel with main conference talks. There are actually more hours of Open Spaces, in total, than there are of talks! While most of the conference is planned months in advance, Open Spaces are created on-site by PyCon attendees. They offer groups the ability to self-gather, self-define, and self-organize in a way that often doesn’t happen anywhere else at PyCon.

Open Spaces are one-hour meetups during the three main conference days, held in meeting rooms within the PyCon convention center. Some people reserve spaces to discuss a favorite technology — like web frameworks, neural nets, or natural language processing — while other people focus an open space on an interest like astronomy, data science, teaching. Other attendees schedule actual activities during Open Spaces, like yoga, nail painting, and board games! Attendees can discover these events via the Open Spaces board which will be next to the registration desk.

Any topic or activity that two or more attendees are interested in could be a good candidate for an Open Space. You can find a list of sample ideas a few pages down in the Open Spaces guide on our web site: https://us.pycon.org/2017/events/open-spaces/

If you have additional ideas, please email us at [email protected] and we can add them to the list.

An extra day to plan each Open Space!

Like last year, each day will feature two Open Space sign-up boards near the registration area: one for the current day, and one board that is already up for the following day.

This will allow hosts to reserve a slot a full day in advance — creating a longer window for them to advertise the space to interested attendees. And attendees will be able to go ahead and start planning which Open Spaces they want to attend the next day.

In fact, the first Open Spaces board will be up on Thursday evening during the Opening Reception, the evening before the main conference even starts! This will give hosts a chance to reserve a slot for the first day of the conference while it is still the night before.

Promote Your Open Space

We are using the hashtag #PyConOpenSpace again this year. We encourage you to use this hashtag to promote your Open Space. It’s also a great idea to add your Twitter handle to the card that you pin on the Open Space schedule board, in case anyone interested in attending your open space has a question or wants to contact you about it.

The committee is looking forward to all of the great Open Spaces that are awaiting us at PyCon US 2017!

May 15, 2017 07:38 PM

PyCharm

PyCharm 2017.2 EAP Starts now

The JetBrains PyCharm team is thrilled today to announce the start of the PyCharm 2017.2 Early Access Preview (EAP) program along with the 2017.2 EAP contest!

PyCharm 2017.2 Early Access Preview (EAP) 1 is finally available for download! Head on over to the PyCharm Early Access Preview (EAP) page and get it now!

What is an Early Access Program (EAP)?

We at JetBrains believe that making tools for developers should greatly involve listening to developers. Our Early Access Programs let the development community closely participate in discussions about the product and influence development planning, from early stages onwards.

Why should you try the 2017.2 EAP builds?

The Early Access Program lets you evaluate brand-new features as well as many important bug fixes for already existing functionality to be added to the next release. You can install PyCharm 2017.2 EAP along with the stable PyCharm 2017.1 installation and get regular updates for both. An easy way to keep your PyCharm up to date is to use JetBrains Toolbox.

Why should you try this build?

@typing.overload is now supported in regular Python files, in addition to the support in Stubs which was already available in earlier versions of PyCharm
SSH-Agent is supported, if you prefer to keep your SSH keys safe using SSH agent (on Linux and macOS) or Pageant (on Windows), this is now supported in PyCharm. Just set authentication type to authentication agent when setting up your SSH connection. [This feature is only available in PyCharm Professional Edition]
Better support for HiDPI setups: we now support having different DPI settings per monitor. For example, when you have a laptop with a high-resolution screen, which you connect to an external monitor.
And more: check out our release notes for details

PyCharm 2017.2 EAP contest: File bugs, feature requests, participate in active discussions and win!

With the PyCharm 2017.2 EAP program we decided to fire up a contest, where everybody is welcome to participate. This is a great chance for professional Python developers to work closely with the PyCharm development team on the next release version. Get your professional development juices flowing: file bugs and feature requests, participate in active discussions, and compete for valuable prizes!

You’ll have about 2 months until the final PyCharm 2017.2 release to participate. All you need to do is just start using EAP builds and when you find a bug, file it, and then participate in later conversation on this and other bugs created by other people.

Filed tickets and comments will be scored on a range of different criterias, as it’s hard to judge based only on the number of bugs filed or comments written.

We’ll take into account:

New bugs in EAP builds
Regressions
Useful comments that help to reproduce or fix a problem
New feature requests and valuable suggestions to improve new or existing features

Of course, the judgment of usefulness is a very subjective thing. The PyCharm team is going to carry out the final scoring at the end of the EAP program. To the most active
EAP users we’ll provide valuable prizes, including:

Professional Python developer t-shirts
Free PyCharm Professional Edition Subscriptions
Free seats at the latest Michael’s Kennedy “Python for Entrepreneurs” course
And more

How to file bugs, score and win

Use our public issue tracker to report a bug, request a new feature, look for existing problems, and comment on existing issues. Using YouTrack, you can look through issues others have reported and vote for them. Whenever your issue or an issue you’ve voted for is updated, you will receive notifications.

If you’d like to communicate with us, you can reach us by leaving a comment on this blog post, by tweeting us at @pycharm or leaving a comment on our Facebook page.

Give PyCharm 2017.2 EAP a try before its official release and don’t forget to report any bugs and feature request to our issue tracker!

May 15, 2017 05:28 PM

Doug Hellmann

gzip — Read and Write GNU zip Files — PyMOTW 3

The gzip module provides a file-like interface to GNU zip files, using zlib to compress and uncompress the data. Read more… This post is part of the Python Module of the Week series for Python 3. See PyMOTW.com for more articles from the series.

May 15, 2017 01:00 PM

Mike Driscoll

PyDev of the Week: Anand Balachandran Pillai

This week we welcome Anand Balachandran Pillai as our PyDev of the Week! Anand is the author of a new book called Software Architecture with Python from Packt Publishing. He is the founder of the Bangalore Python Users Group (BangPypers) and a member of the Python Software Foundation. Let’s take a few moments to get to know Anand better!

Can you tell us a little about yourself (hobbies, education, etc):

I have done my engineering in 1998 – in Mechanical Engineering from the Indian Institute of Technology at Chennai. However I have never been a mechanical engineering even for a day in my life, as immediately after graduation I took up my first job in computer software.

Hobbies – I’ve been an avid trekker in the past, I still like to do it sometimes. Though more regular hobbies are the usual – listening to Music, Reading and going for long drives.

I’ve been an avid fan of Sherlock Holmes from younger days at school. A minor hobby is collecting Holmes stories and books in various languages. I even try his theories in real life on friends and family with variable success!

Why did you start using Python?

I first started using Python around October 2002 when I was working with a company in Bangalore as software engineer.

They were using a proprietary language – something more than C, but less than C++. I was then 5 years into my career and was getting frustrated at just knowing C and C++ and not really getting a chance to use the latter as well.

I wanted to learn something outside C/C++. I tried Perl for a while but got lost in its arcane syntax and gave up after a month. I somehow happened to come across the Python website at that time. As it says in the official tutorial (the old one – it was called “Learn Python in an afternoon” at that time!)- I literally tried it first in an afternoon – and learned it then and there.

I used nothing but the official tutorial and the official module documentation to learn Python in the initial few months.

Later on I bought a copy of “Python in a Nutshell” by Alex Martelli – and it was my Python bible for the next 2-3 years.

Note that I never used Python for any official work for another 3 more years. I mostly spent time developing hobby projects in Python – the most productive being HarvestMan – the web-crawler I wrote in Python myself.

What other programming languages do you know and which is your favorite?

I can program any day in C, having done it for more than 10 years in college and career, though my C++ is a bit rusty now.

I know Perl pretty well – I was destined to meet up with it again during my time in Yahoo! (2010 – 2012) where I wrote a lot of code in it, having lost my earlier repulsion for its syntax. I did find it a pretty productive language at that time – though once I left Yahoo!, I haven’t used it in any personal projects.

At Yahoo! I also programmed in PHP – there was no way around it – though I wouldn’t touch it with a flag-pole now if possible.

I tried to learn Ruby – but frankly I didn’t like the language as knowing Python and Perl – I guess I couldn’t find the right place for Ruby in the scheme of things in my mind – so I left any attempts to pick it up. My most recent job was in a company that used Ruby for most of its work – though I never programmed a single line in it over there. I preferred to work using Python!

These days I have been spending time trying to pick up Rust and also did an online course on Erlang.

I liked the way Erlang expresses logic using patterns – it felt very symbolical and expressive when compared to anything I have learned so far. However I have never used it for any projects – yet.

What projects are you working on now?

For my current company, I am working on writing scalable crawlers – so a lot of time is spent on Scrapy and writing custom crawlers using it.

I do contribute a bit to open source projects. Recently I’ve been spending some time on Junction – the conference software that we built in the Python community in India – for powering PyCon India and similar websites – helping fix issues and mentoring some students to contribute patches.

Another goal of mine this year is to contribute more patches to CPython – I have done around 3 or 4 so far since 2008 – but I want to do more serious work there and take it up as a more or less regular activity within the next couple of years – and hopefully mentor young college graduates to contribute patches to CPython.

Which Python libraries are your favorite (core or 3rd party)?

Core – I’d say multiprocessing. This is something which I’ve been using more recently in work – to split data parallel computations across cores. I especially like the Pool object!

3rd party – Many libraries, though something I’ve been impressed with would be SQLAlchemy, considering how powerful it is and mostly been developed by just one person (Mike Bayer). I have given professional trainings on it but it never ceases to amaze me with its repertoire of tricks and recipes.

What made you decide to write a book about Python?

I’ve developed a rich and (perhaps) unique set of experiences – as an architect and as a Python programmer over the last 10-15 years of using Python.

I wanted to share my experiences in a book form – for some time. I thought of combining the way an architect approaches problems with the Python way of solving things and came up with an idea of writing a book on Software Architecture using Python.

Another aspect has been my experience with using Python for writing design patterns in a unique way.

Python can be used to write design patterns in very different ways when compared to less expressive languages like C++ or Java. One can get very creative with Python when writing patterns.

In my book I’ve tried to illustrate this in many examples on the chapters on Design and Architecture patterns – there is no need to get pedantic with Python and patterns but instead one should let his/her creativity flow!

This spirit is best illustrated by Alex Martelli’s Borg (non-pattern ?) pattern which I’ve explained in detail in the book. For examples of how to get creative one can look at the way I mix Singleton and Prototype in one pattern using Meta-classes plus my implementation of the State pattern using Python iterators – in the book.

The book URL is here: https://www.packtpub.com/application-development/software-architecture-python

Who is the target audience?

People who’ve been doing Python for more than 5 years and is getting used to its expressiveness but aren’t still sure of themselves where to go next.

People who have been exploring Python and its power – in terms of meta-classes and generators and the like – but need a guiding hand in terms of structuring their code around patterns and building the right DNA in their products towards software architectural quality attributes.

People who are in either of these categories above -aspiring to be Software Architects – and would like to express themselves – using Python in a better and more structured way.

What did you learn while writing the book?

I learned a lot about structuring my thinking right. I learned how to take feedback in the right way and incorporate into my work. I believe I have become more open minded as a result of this work – having worked with a great team of reviewers and editors.

I think the code examples I’ve given in the book – many of them came with a lot of hard work and thought about the problems and creative approaches to solving them. I myself learned a lot about concurrency and scalability when writing the book – and clarified a lot of my own misconceptions when doing research.

And thanks for reviewing my book Mike

Is there anything else you’d like to say?

I encountered Python in 2002 and it’s been nearly 15 years in its company. The impact Python have had in my life is tremendous. There are lifelong friends I have made, journeys I have undertook and paths I took – because of Python.

For people who want to use and contribute to an open source language – I would say give Python a try! The community is very welcoming, the language is scaling new heights – by way of its adoption in data analytics and machine learning.

Overall for me it has been a rich and rewarding journey – and a pleasant one too.

Thanks so much for doing the interview!

May 15, 2017 12:30 PM

Planet Python

May 18, 2017

When to Use Python

Python is a great“adjustable desk” language.

Now, is using Pythonalways the right choice?

By learning Python you’re not limiting yourself to a specific niche.

May 17, 2017

Help!

It’s Easy

The White Album

Do You Want To Know A Secret?

Mr. Post(gres)Man

Get Back

Full Speed Ahead, Mr. Boatswain!

The End

More sessions to come

What

Lightning talks!

Mailing List

May 16, 2017

Creating a Simple Tree

Creating an XML Viewer

Other Odds and Ends

Wrapping Up

Related Reading

The Problem

The Python Nonsense

The code with a -2/10 pylint score

Addition is Commutative

Permutations

The n things taken r or 2 at a time problem

Another Perspective

The Worst Code Snippet Ever

The Big-O Problem

The Binary Search

Added Complication

The Kicker

Lessons Learned

Stacks in Python

✅ The list Built-in

✅ The collections.deque Class

✅ The queue.LifoQueue Class

A good default choice: collections.deque

Lazy Sequences working hard

May 15, 2017

What are Open Spaces?

An extra day to plan each Open Space!

Promote Your Open Space

What is an Early Access Program (EAP)?

Why should you try the 2017.2 EAP builds?

Why should you try this build?

PyCharm 2017.2 EAP contest: File bugs, feature requests, participate in active discussions and win!

How to file bugs, score and win

Python is a great
“adjustable desk” language.

Now, is using Python
always the right choice?

A good default choice: `collections.deque`