Planet Plone

This is where developers and integrators write about Plone, and is your best source for news and developments from the community.

November 03, 2016

Reinout van Rees: DSA ssh keys also deprecated in OSX Sierra

by Reinout van Rees at 2016-11-03T08:05:00Z

I've been using an ssh key for a long time. Back in the days when RSA keys were mostly commercial/proprietary and when DSA keys where the recommended method.

When ubuntu 16.04 came out, I suddenly couldn't use my key anymore as ubuntu (or perhaps rather its updated ssh server) didn't accept DSA keys anymore. They're apparently not save enough anymore.

I worked around it by adding a setting on the server, telling it to accept my DSA key.

When the latest Apple OS update (Sierra) came out, I suddenly couldn't log in anywhere. Also git pull to github (I'm using ssh) stopped working. I found the cause by passing -vvvv to ssh:

debug1: Next authentication method: publickey
debug1: Trying private key: /Users/reinout/.ssh/id_rsa
debug3: no such identity: /Users/reinout/.ssh/id_rsa: No such file or directory
debug1: Trying private key: /Users/reinout/.ssh/id_ecdsa
debug3: no such identity: /Users/reinout/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /Users/reinout/.ssh/id_ed25519
debug3: no such identity: /Users/reinout/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password

Oh.... Sierra's ssh client also doens't like DSA anymore. Ok... time to create a new key :-)

... which I did about 6 weeks ago. With a nice, long passphrase. And 2 weeks ago I had to create a new one as I couldn't for the life of me remember the exact passphrase. I've rarely felt so stupid :-) A passprase can have many almost-right variants:

I'll take the train to django under the hood
I'll take the train to django under the hood.
I'll take the train to Django under the Hood
I'll take the train to Django under the Hood.
I will take the train to django under the hood
I will take the train to django under the hood.
I will take the train to Django under the Hood
I will take the train to Django under the Hood.
I'm taking the train to Django under the Hood.
I'm taking the train to Django under the Hood
I'm taking the train to Django Under The Hood.
I'll go by train to Django Under The Hood.
etc...
etc...

Well, everything works again :-)

November 02, 2016

Gil Forcada: Easily extract commits from a messed up branch

by gforcada at 2016-11-02T13:40:43Z

Once in a while you encounter branches that are have been over-used, i.e. multiple persons added commits there for unrelated issues.

How to get around/solve that? I’m glad you ask :-)

Today I was facing something similar at work, where the messed up branch in question had some commits already merged into master but there were some important other parts to be extracted from it.

So I used three tools:

  • git log --graph --decorate --pretty=oneline --abbrev-commit --all (with an alias git fulllog)
  • a git graphical visualizer (gitg for instance)
  • a simple plain text editor (gedit for instance)

Workflow

Fire up git fulllog in a terminal and copy & paste as much information as you need into the text editor.

Fire up the git graphical visualizer and check each relevant commit what’s doing.

Once you know what it is about annotate each commit so you know which commits relate together, i.e. from

ae0ee04 My commit message

to

ae0ee04 11111 My commit message

This way once everything is annotated is really simple to just grab all the 11111 commits and git cherry-pick them in a new branch :-)

Pro tip: as you are editing a plain text file you can keep removing lines and adjusting the indentation, suddenly you realize how things keep fitting together!

Done!

David "Pigeonflight" Bain: Letsencrypt, Apache2, Plone and SSL

by David Bain at 2016-11-02T13:26:00Z

These are notes for my future self. I'm mostly using nginx now, but I have an Apache server here and there. I also have a
These are notes for my future self. I'm mostly using nginx now, but I have an Apache server here and there. I also have a post on configuring letsencrypt SSL with nginx.

Assumptions

This is a Plone site which uses a proxypass style rewriterule. The site is located at the root of your Zope application server and it is called "Plone", you're using letsencrypt with certbot to generate SSL certificates. Your .well-known folder (used by letsencrypt) is located at /var/www/letsencrypt.

The implementation

With these things in place I found that I needed to precede my standard Plone rewrite rules with a rewrite rule to serve the contents of the letsencrypt .well-known folder. In the examples below my site is running on port 8080, this may be different for you.

Add this so that you an successfully request an SSL certificate via letsencrypt over http.

RewriteRule ^/\.well-known/(.*) /var/www/letsencrypt/.well-known/$1 [L]
RewriteRule ^/(.*) \       http://localhost:8080/VirtualHostBase/http/{HTTP_HOST}:80/Plone/VirtualHostRoot/$1 [L,P]

Add this AFTER you have a working SSL certificate installed and you're already serving stuff over https (see below)
RewriteRule ^/\.well-known/(.*) /var/www/letsencrypt/.well-known/$1 [L]
RewriteRule ^/(.*) \       http://localhost:8080/VirtualHostBase/https/{HTTP_HOST}:443/Plone/VirtualHostRoot/$1 [L,P]


Installing a certificate

Assuming you've done everything above, you can install a certificate from letsencrypt with the following instructions:

Step 1 - Install Certbot


The part of the documentation that I read was silent about where to put the certbot-auto script. I decided to install certbot-auto in /usr/local/sbin, this means that it is in the system path and can be run as a command by itself (which feels nicer than needing to be in the folder when running the script).
cd /usr/local/sbin/
sudo wget https://dl.eff.org/certbot-auto
sudo chmod a+x certbot-auto

Step 2 - Request a certificate against the /var/www/letsencrypt 

Now you can easily request a certificate.

Note the use of --webroot in the command below, this ensures that the challenge information is added to the /var/www/letsencrypt folder. Also, I precede the command with sudo because installation of the actual certificates requires admin privileges.
sudo certbot-auto certonly --webroot -w /var/www/letsencrypt -d myproject.example.com
Expected response
If everything was done properly you'll get a message like this:

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/myproject.example.com/fullchain.pem.
   Your cert will expire on 2016-11-25. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot-auto
   again. To non-interactively renew *all* of your certificates, run
   "certbot-auto renew"

Step 3 - Configuring Auto-renewal of certificates

Once setup properly the auto renewal steps that I took were near identical to the documentation.
I first tested to see that an auto renewal would work without issue:

certbot-auto renew --dry-run

Once the dry run was succesfull I added a renewal command as a cronjob using the crontab -e command:

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
@monthly certbot-auto renew --quiet --no-self-upgrade

Additional notes

I've read places that say that mod_alias always get's precedence over mod_rewrite, so I tried to achieve this using mod_alias initially. That didn't work for me so I settled on this approach.

November 01, 2016

Makina Corpus: The world's simplest Python template engine

2016-11-01T00:00:00Z

A template engine is a component able to render some data using a given string template.

We use it extensively in web development (that's not surprising because web development is mainly about reading some data and render them as strings, that's what I say when someone ask me about my job).

So there are already a lot of Python based template engines (Jinja2, Mako, Chameleon, ...).

The purpose of this article is to build a very basic template engine based only on the Python standard string class formatting capacities.

You probably know...

You probably know .format() replaces the old "%" based formatting solution:

>>> "My name is %s and I am %d" % ("Eric", 41)
'My name is Eric and I am 41'
>>> "My name is {name} and I am {age}".format(name="Eric", age=41)
'My name is Eric and I am 41'

It allows to perform all the typical formatting features, for instance:

>>> '{:^20}'.format('centered')
' centered '
>>> '{:^20}'.format('align right')
' align right '
>>> '{:>20}'.format('align right')
' align right'
>>> '{:<20}'.format('align left')
'align left '

and many other things (have a look to https://pyformat.info/).

Did you know?

format() is also able to access attributes or items of the parameters.

For instance with a dictionary, we can do this:

>>> 'Name: {person[name]}, age: {person[age]}'.format(person={'name': 'Eric', 'age': 41})
'Name: Eric, age: 41'

And the same goes with attributes:

>>> class Song(object):
... title = 'Where is my mind'
...
>>> 'My favorite song is: {song.title}'.format(song=Song())
'My favorite song is: Where is my mind'

That's really cool. It starts looking like a template engine, right?

Ok, but few things are missing

What we usually expect from a template engine is to be able to:

  • call methods,
  • make loops over iterables,
  • manage condition.

Let's see how we can handle that.

Calling methods

A method is an attribute of an object, we can call attribute, why couldn't wecall a method? Let's try:

>>> 'My name is {name.upper}'.format(name='eric')
'My name is <built-in method upper of str object at 0x7f67dc8d1630>'

Yeah, not exactly what we expected...

An interesting feature of format() is the format specification: instead of just inserting a field with {field}, we can specify a format like this: {field:spec}.

That's exactly what we do with float for instance:

>>> "{:.3}".format(3.14159)
'3.14'

Well, it is actually very easy to implement our own spec by derivating the Formatter class. So let's implement a ':call' spec in charge of calling the current field:

class SuperFormatter(string.Formatter):
    def format_field(self, value, spec):
        if spec == 'call':
            return value()
        else:
            return super(SuperFormatter, self).format_field(value, spec)

We can use it that way:

>>> sf.format('My name is {name.upper:call}', name="eric")
'My name is ERIC'

Nice!

Loops

Similarly, we can implement a :repeat spec,

class SuperFormatter(string.Formatter):
    def format_field(self, value, spec):
        if spec.startswith('repeat'):
            template = spec.partition(':')[-1]
            if type(value) is dict:
                value = value.items()
            return ''.join([template.format(item=item) for item in value])
        else:
            return super(SuperFormatter, self).format_field(value, spec)

Here, we pass a parameter to the spec to provide the template to use when we iterate on the loop items. So the resulting format is: <field>:repeat:<template>.

This subtemplate is a regular format() template where we escape the curly brackets by doubling them, and where the only field is the loop variable (named item).

So we can use it like this:

>>> sf.format('''Table of contents:
... {chapters:repeat:Chapter {{item}}
... }''', chapters=["I", "II", "III", "IV"])
'''Table of contents:
Chapter I
Chapter II
Chapter III
Chapter IV
'''

Condition

Let's also implement a :if spec to test the field value, and then display or not the subtemplate:

class SuperFormatter(string.Formatter):
    def format_field(self, value, spec):
        if spec.startswith('if'):
            return (value and spec.partition(':')[-1]) or ''
        else:
            return super(SuperFormatter, self).format_field(value, spec)

At first, it seems stupid, because it looks like it will only be able to conditionnally display static portions, like this:

>>> sf.format('Action: Back / Logout {manager:if:/ Delete}', manager=False)
'Action: Back / Logout '
>>> sf.format('Action: Back / Logout {manager:if:/ Delete}', manager=True)
'Action: Back / Logout / Delete'

What if we want to render conditionnally a portion of template containing fields, like this:

>>> sf.format('Action: Back / Logout {manager:if:/ Delete {id}}', manager=False, id=34)
'Action: Back / Logout '
>>> sf.format('Action: Back / Logout {manager:if:/ Delete {id}}', manager=True, id=34)
'Action: Back / Logout / Delete 34'

Hey that works! That was unexpected. To be honest, I first wrote the test exactly like this, and I expected it to fail, but it was not! Why is that?

Because here the curly brackets are not escaped, so they are processed by the main format() call. So cool!

Here we go

That's it, we have a pretty nice and complete template engine.

The full code of our world's simplest Python template engine is here, the implementation itself is 10 lines long, Python is a really powerful language!

If you have any funny idea to improve it (remember, I want to keep it short), pull requests are welcome.

The objective is mostly to have fun with Python and demonstrate what the standard Formatter class is able to do, but, having a second though, I might intregate it in Rapido :).

Makina Corpus: Paypal tracking with Rapido

2016-11-01T00:00:00Z

Paypal Instant Payment Notification (IPN)

If we put a Paypal button somewhere in a website, it is nice to know if the user got through the payment process properly and did not just canceled in the middle.

To achieve this, Paypal provides an asynchronous messaging service named Instant Payment Notification (IPN). Here is the process:

  • we create a regular Paypal button and we add an extra hidden input named "custom" which value will be our tracking id (it can be the user id, or anything relevant to our case),
  • when the user clicks on the button, he/she leaves our site and goes to Paypal, and once the payment is complete, Paypal notifies us by making a request to our IPN URL (this is an URL we declare in our Paypal settings),
  • this request contains all the payment information including the custom variable (so we can know who paid what, how much, and store that in our system),
  • to acknowledge the notification, we are supposed to do 2 things:
    • reply with an empty content
    • make a POST containing the very same information contained in the notification request plus "cmd=_notify-validate"
  • and then Paypal answers to our POST by returning "VERIFIED".

How to do it with Rapido

Creating the Paypal button is easy, we just copy/paste the core provided by Paypal in a block (see the Rapido documentation to learn how to create blocks) and we add our "custom" variable:

pay.yaml:

elements:
    trackingid:
        type: BASIC

pay.py:

def trackingid(context):
    currentUser = context.api.user.get_current()
    return currentUser.getUserName() # or anything else relevant

pay.html:

<form action="https://www.sandbox.paypal.com/cgi-bin/webscr" method="post" target="_top">
    <input type="hidden" name="custom" value="{trackingid}"/>
   <input type="hidden" name="cmd" value="_s-xclick">
   <input type="hidden" name="hosted_button_id" value="XXXXXXXXXX">
   <input type="image" src="https://www.sandbox.paypal.com/fr_FR/FR/i/btn/btn_paynowCC_LG.gif" border="0" name="submit" alt="PayPal, le réflexe sécurité pour payer en ligne">
   <img alt="" border="0" src="https://www.sandbox.paypal.com/fr_FR/i/scr/pixel.gif" width="1" height="1">
</form>

Note: make sure we do not insert the full block form but just its children with our Diazo rule because we would get a form into a form, so the rule will be like that:

<after css:content="h1.documentFirstHeading">
<include href="@@rapido/myapp/block/pay" css:content-children="form[name='pay']"/>
</after>

Now let's see how to implement the listener that will get the Paypal notification. At some point, we will need to create a POST requests, so we need an extra library for that. We will use the well-known Python requests library, so we need to declare it safe for Rapido.

Hence we add the following somewhere in our custom theme module, for instance in __init__.py (or anywhere else):

import requests
from rapido.core import app
app.safe_modules.requests = requests

Warning: allowing to use a module like Requests in Rapido can have consequences regarding security, so if you do not control who is allowed to create Rapido apps (by default, only managers), do not do it.

And now, we create a block named ipn, containing an element named listener, the IPN URL we will declare to Paypal will be:

http://my.server.com/@@rapido/myapp/block/ipn/listener

Here is the implementation:

ipn.yaml:

elements:
listener:
type: BASIC

ipn.py:

def listener(context):
trackingid = context.request.form['custom']
paypal_params = {'cmd': '_notify-validate'}
# collect all the Paypal params
for key in context.request.form:
paypal_params[key] = context.request.form[key]
# acknowledge with a POST
req = context.modules.requests.post(
'https://www.sandbox.paypal.com/cgi-bin/webscr',
data=paypal_params)
# check if everthing is ok
if req.text == "VERIFIED":
mark_payment_has_done(trackingid)
# return an empty answer to the Paypal original request
return ''

That's it, that's just 10 lines of code (of course we will need to implement mark_payment_has_done() depending on our case inb order to persist in a record that the payment is done and verified). 

October 30, 2016

Patrick Gerken: Jonathan Stoppani - Managing dependencies of Python projects pyconde16

by Patrick Gerken at 2016-10-30T00:00:00Z

Assumptions:

  • You use isolated envs
  • You use pip
  • You work on products, not libraries

Part 1

You start with::

pip install django
pip install boto...

Now lets deploy. SSH to server, copy dist files, copy source, talk done!

Second deployment:
O, missed dependencies. requirements.txt to the rescue

Third deployment:
Everything breaks again, because you did not pin your packages yet.

Fourth deployment:
Also pin dependencies of your dependencies:

pip freeze > requirements.txt

pip does not check that your dependencies are consistent to each other.

pip-tools helps with that.

pip-compile recursively compiles dependencies together and checks for consistency

Workflow:
Create a requirement.txt.in with top level requirements.
Run pip-compile, and the requirements.txt contains all pinned requirements.

Every time you run pip-compile, the second level dependencies get updated again. If after that, tests fail, you must update your requirements.txt.in to document the known incompatibility.

Part 2

Number:

  • 5k websites,
  • 400 daily deployments.
  • 2 indexes (devpi)
  • 15 top level and 120 total distributions
  • Goal: deploy in less than 2 minutes.

pip compile takes time and can be platform specfic. pip install takes time to resolve distributions, resolves depdencies and install the distributions.

Our solution: wheelsproxy a pypi proxy, builds wheels, knows abouts platform. http://github.com/divio/ac-wheelsproxy

Uses docker to build wheels for the different platforms.

pip install went down from 180 seconds to 15 seconds.

Demo time: reqs.in file with 15 requirements. uploads reqs.in to wheelsproxy, then pip compile runs on the server with cached data. pip installed django with 120 requirements in 40 seconds.

Summary:

  • use pip-compile (thanks @nvie)
  • Have test coverage to know that updated packages will work
  • Cache your built wheels
  • Scale up with the wheelsproxy

Questions: What about tools to see if there are newer versions.
Answer: you committed the reqs.txt, so run git-compile again and check the diff.

Patrick Gerken: Andreas Jung - Hunting for the best NoSQL database. Why we love ArangoDB

by Patrick Gerken at 2016-10-30T00:00:00Z

NoSQL is not about performance, scaling, dropping ACID or hating sQL, it is about choice.

Categories of NoSQL:

  • key-value
  • column oriented
  • document based
  • tabular
  • graph
  • xml/object db

New challenges

  • cloud
  • replication
  • data explosion (big data)
  • globally distributed systems
  • specialised requirements

For this we need new solutions (nosql)

CAP Theorem

  • Consistency
  • Availabilty
  • Partition tolerance

Choose two (or less)

My personal hunt for a multi purpose nosql db

  • Should fit most mid size projects
  • document store
  • arbitrary queries
  • cross table relationships
  • Transactional integrity, ACID, optional
  • replication/clustering

FoundationDB

Got bought by Apple and killed

MarkLogic

Highend expensive solution. higly professional, feature complete, saved Oama Care

ArangoDB

A multi model database:

  • JSON
  • Key Value
  • Graph database
  • models can be combined

Foxx framework

  • implement you own rest micro services directly in javascript inside arango db
  • batteries included, job queue

AQL

  • declarative human readable DSL for queries
  • Query both documents and graphs
  • ACID Suppport
  • Easy to understand if you know sql. (Now a code sample)

Clustering, replication, sharding

  • Agency (cluster control via raft consensus protocol)
  • synchronous/async replication with automatic failover

Benchmarks

Lots of hard to read images showing that Arango is quite fast

Python bindings

Hard to read again, but apparently you can treat a Arango document a bit like a dict.

Misc

  • Current version 3.0
  • good documentation
  • regular updates
  • good community support.
  • supported by ArangoDB Gmbh in Cologne (\o/)

October 29, 2016

Patrick Gerken: Lightning Talks Saturday

by Patrick Gerken at 2016-10-29T00:00:00Z

Alex Hendorf

First Europython 2017 Announcement.
We had lots of proposals.
We choose Rimini.
Lots of applause after some nice pictures.
Conference center got awarded as best Conference center.
Good Hotel rates. No Airport in Rimini, come via Bologna.
Like last year in July.
http://www.europython.eu

Reimar Bauer

Pythoncamp 8.- 9. April 2017
Cologne, Free entrance.
http://www.pythoncamp.de
Different to conference
Everybody should give a talk.
Last year 70 attendees and 60 talks.
Need sponsors.
Need organizers.

Daniele Procida

Wanted a Python adventures in Africa.
Had a dream of doing something with python in africa.
python Namibia 2015
Python Namibia 2016
It is easy to get to Namibia. From Frankfurt go to south afrika, shortly
before, turn right.
Venue in Windhoek.
118 attendees, 50% women (lost the rest from the slides)
4 days program
Many sponsors (Django)
PSF supported with 50 preconfigured Raspberry Pi
Got a lot of press coverage, well received.
As a result now also a Pycon Zimbabwe 2016

Steffen Allner

Zope's not dead
We talked with Plone. We don't want it dead.
Would like to move to Python 3. 30 people in a sprint.
People use it: In routers, for other CMS than Plone...
Successful sprint in Halle.
RestrictedPython is main difficulity for Python 3.

Roadmap:

  • Q1.2017 RestrictePython
  • Q2. Working Zope app
  • Q3 2. Resurrection spring
  • Q4. Zope Release with Py3

Join us. http://www.meetup.com/de-DE/Zope-Sprint/

Claus Aiohinger

Announcement Pycon2017 Slovakia. 10.3.-12.3. 2017 in Bratislava

CFP is still open. Looking for first time speaker specifically

Please come!

coala Team

Coala. Tradtion in giving Lighting talk.
Coala for linting and fixing every programming language

Would you want to rewrite Libreoffice to add spellchecker for another language?
Then why do you do it with linting tool?

Coala provides a framework to reuse linting tools for many languages.
Support 56 languages. 57 since 5 minutes.
We wrap existing tools.

Coala can not only find problems, but also fix them.

Got something new: Moving from declaring the tools to use for checking to declaring what type of checks to run, independent of programming languages. We are looking for sponsors.

http://coala.io/usability
Win a shirt, beer or chocoloate.

Max Scholz

(This is a bit incomplete, the speaker spoke in long sentences that are hard to follow if you try to summarize them in parallel. please watch the talk to get more sense out of it)

I told the story already at europython.
It is about me trailing in sweden.
3 years ago I needed a break from university and humans.
It was the first time I walked, it was new for me to be alone. I took a camera and made pictures. Most of my memories were pictures.
2 years ago I went again. This time, instead of taking pictures of things I liked, I just sat down and enjoyed it.
One time, I sat for 2 hours and was suddenly full of flies. Why I am the king of flies I thought. Maybe because I was sweating?
My gist: If you go for a walk, take your time, don't waste time making pictures, enhjoy your leisure time to the fullest and just watch.
My difference to europython conference talk:
I did not want to emphafize to never take pictures. I don't want to document what I am doing, I just want to enjoy what I am experiencing.
Be mor mindful about what you experience.

Christian Barra

PyData Poland Announcement
First national PyData Conference.
Topics:
PythonR Julia Databases Machine learning npp....
3 day conf Friday to Sunday
Probably during Spring 2017
Nno definitive location yet. Somwhere in Poland.
http://www.pydata.pl (Not working yet)
Who wants to join?
We aready have a CoC.
We want to have a track: Women in pydata.

We need sponseors, co-organizers, speakers...

Mike Muller

PySV
Serving the german speaking python community
pysv.org
Representing the community
organizes as german foundation
Mike Muller Reimar Bauer Thomas Fabula are in the current Board
We support Usergroups, Events, Public Relations.
Or help is ogranisational, financial, technical (Infrastructure)

We are organized in working groups:

  • PyConDE
  • Public Relations
  • external networks
  • communities...

Coordination happens via monthly irc meetings.
Open for all.

Become a member:

  • Personal member
  • freelancer
  • Companies

General Assembly: Today 18:00 room Omega (Board Election)

Personal statement:
As part of the plonekonf2012 Orga Team, I can testify that there would not have been a conference without the support from PySV. Thank you!

Anselm Kruis

State of Stackless python
Today I tagged Stackless 2.7.12.
No funding for mainting Python 3, 3.3 works, 3.4 has bugs, 3.5 crashes
Volunteers are welcome
Join us: https//bitbucket.org/stackless-dev/stackless

Martin Christen

Geo python 2017
May 8-10 in Basel, Switzerland
Call for Contributors
Last event in June
http://2016.geopython.net
This year
http://2017.geopython.net

GIS, Big Data... Machine Learning

@GeoPythonConf

Benedikt Eggers Markus Schaber

Ironpython
We are working on it again, after it turned dead, after microsoft dropped it.
Python written in native C#
Complete integration, most of python stdlib is there.
You can use .netlibraries/assemblies together. Just import them as if they were python
(Code example)
Writing class in C#, in direct import in IronPython
Ironpython is not dead any more
10 active contributors.
Now works on all(?) .net platforms, not only windows.

Anreas Kopp

Hackerspace: http://www.erfindergarden.de Munich, Kolumbuspplatz

  • Python
  • 3dprinting
  • rasbperry pi
  • lasercutter
  • fun
  • play Planned to be a non profit.

Fab Academy 2016 a 5 month program You get a diploma from Chile. fabacademy.org

Emil

Our conf wifi.
Shows how to install something for the wifi. But why?
Some live hacking for the install script.
Oh, he is making a live demo with a non licenced sublime.
So my seating neighbors told me the script was supposed to install the required certificate to make the Enterprise WPA Login work.

Orga team

Best image tweet
(Hope it will be mine)
Not the python conference you were looking for
Ok, I won a quadrocopter, anybody has 4 AAA batteries?
Thank you!

Now beer

October 27, 2016

CodeSyntax: World Plone Day 2014: manufacturing internationalization strategy

2016-10-27T09:46:03Z

World Plone Day was last wednesday, april the 30th, and as in previous occasions, we did celebrate it at CodeSyntax's offices, with some customers and Plone users of the Basque Country.

October 26, 2016

Gil Forcada: Plone testing team meeting

by gforcada at 2016-10-26T17:14:26Z

After a Plone Conference there’s always a lot of energy and willingness to move Plone one step forward.

The testing and CI team in Plone is no different, and for that I created a Doodle to see who is interested and what would be the best time to meet during next week.

If you are interested in the team, or know anyone who would be, please share that link and join the meeting!

October 25, 2016

Mikko Ohtamaa: Deform 2.0

by Mikko Ohtamaa at 2016-10-25T23:22:39Z

Deform 2.0 has been released.

Deform is a Python form library for generating HTML forms on the server side. Date and time picking widgets, rich text editors, forms with dynamically added and removed items and a few other complex use cases are supported out of the box.

Deform integrates with the Pyramid web framework and several other web frameworks. Deform comes with Chameleon templates and Bootstrap 3 styling. Under the hood, Colander schemas are used for serialization and validation. The Peppercorn library maps HTTP form submissions to nested structure.

Although Deform uses Chameleon templates internally, you can embed rendered Deform forms into any template language.

1. Use cases

Deform is ideal for complex server-side generated forms. Potential use cases include:

2. Installation

Install using pip and Python package installation best practices:

pip install deform

3. Example

See all widget examples. Below is a sample form loop using the Pyramid web framework.

https://github.com/Pylons/deform/raw/master/docs/example.png

Example code:

"""Self-contained Deform demo example."""
from __future__ import print_function
from pyramid.config import Configurator
from pyramid.session import UnencryptedCookieSessionFactoryConfig
from pyramid.httpexceptions import HTTPFound
import colander
import deform
class ExampleSchema(deform.schema.CSRFSchema):
    name = colander.SchemaNode(
        colander.String(),
        title="Name")
    age = colander.SchemaNode(
        colander.Int(),
        default=18,
        title="Age",
        description="Your age in years")
def mini_example(request):
    """Sample Deform form with validation."""
    schema = ExampleSchema().bind(request=request)
    # Create a styled button with some extra Bootstrap 3 CSS classes
    process_btn = deform.form.Button(name='process', title="Process")
    form = deform.form.Form(schema, buttons=(process_btn,))
    # User submitted this form
    if request.method == "POST":
        if 'process' in request.POST:
            try:
                appstruct = form.validate(request.POST.items())
                # Save form data from appstruct
                print("Your name:", appstruct["name"])
                print("Your age:", appstruct["age"])
                # Thank user and take him/her to the next page
                request.session.flash('Thank you for the submission.')
                # Redirect to the page shows after succesful form submission
                return HTTPFound("/")
            except deform.exception.ValidationFailure as e:
                # Render a form version where errors are visible next to the fields,
                # and the submitted values are posted back
                rendered_form = e.render()
    else:
        # Render a form with initial default values
        rendered_form = form.render()
    return {
        # This is just rendered HTML in a string
        # and can be embedded in any template language
        "rendered_form": rendered_form,
    }
def main(global_config, **settings):
    """pserve entry point"""
    session_factory = UnencryptedCookieSessionFactoryConfig('seekrit!')
    config = Configurator(settings=settings, session_factory=session_factory)
    config.include('pyramid_chameleon')
    deform.renderer.configure_zpt_renderer()
    config.add_static_view('static_deform', 'deform:static')
    config.add_route('mini_example', path='/')
    config.add_view(mini_example, route_name="mini_example", renderer="templates/mini.pt")
    return config.make_wsgi_app()

This example is in deformdemo repository. Run the example with pserve:

pserve mini.ini --reload

4. Status

This library is actively developed and maintained. Deform 2.x branch has been used in production on several sites for more than two years. Automatic test suite has 100% Python code coverage and 500+ tests.

5. Projects using Deform

 Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

October 24, 2016

David "Pigeonflight" Bain: Characteristics of a sustainable buildout

by David Bain at 2016-10-24T13:12:00Z



Buildout is a software build system used to manage development and deployment setups, especially in Python. When done well, Buildout makes everything very repeatable. Widely used in Plone projects but not exclusive to such projects, I've used buildout with Pyramid, Tryton and even for Sphinx based documentation projects. I've also seen buildouts on github for Django, Flask and Odoo based projects. 

What I expect of a proper build system

  • Easy to switch between development and production
  • Easy management of packages and package versions
  • Easy to share with other developers
  • A workflow as close to what developers expect

Here's a breakdown of how we do our buildouts to ensure that these goals are met.

The anatomy of a sustainable buildout

There are a few things that I add to my buildout to help to keep things sane. The profiles and templates folder and a requirements.txt file. I discuss their usage below.

Profiles folder

The profiles folder holds the configuration files used by the buildout. Generally the ones that we link to directly are dev.cfg and prod.cfg. These then link to the supporting configuration profiles.


Templates folder

Templates are stored in the templates folder, these templates are used to dynamically generate configuration settings. Some of them include variables which are configured in the .cfg files found in the profiles folder.


requirements.txt to define build tools

I recently added the requirements.txt file to make the workflow closer to a more typical Python development workflow, for example most Flask developers expect such a file. Since Buildout is your build tool, we define it as a dependency in the requirements.txt so that a user can easily install it.

Getting started is a matter of running the well known "pip install -r requirements.txt" command. This installs the buildout command.

buildout.cfg is not stored by git

With this approach the buildout.cfg file is not stored in the repository, so it is added to the .gitignore file. The process of kick starting things requires that a user copies the buildout template from the templates folder into the root of their buildout.

Bootstrapping

Here's a quick example with a real project, the Python Jamaica website. I've added comments to describe each step of the process. I only assume that you are familiar with virtualenv and pip.

 # clone the site, enter the newly cloned directory
git clone https://github.com/PythonJamaica/pythonjam.site
cd pythonjam.site
# create a buildout.cfg file (buildout expects this to be the default config file)
# create a local virtual environment then install the requirements using that environment's pip
cp templates/buildout.cfg.in buildout.cfg
virtualenv venv
venv/bin/pip install -r requirement.txt
# bootstrap the buildout
venv/bin/buildout bootstrap
# initiate a build
bin/buildout
One gotcha to look out for, after bootstrapping a local command bin/buidout is used to run the actual build (not to be confused with venv/bin/buildout).

Thanks to Maik Derstappen for pointing me towards "buildout bootstrap" which replaces the older approach of using a special bootstrap.py file.

Day to day usage

Once the buildout is installed here are some day to day tasks that a developer might find themselves doing.

Launching an instance

During development a standalone instance can be launched with the command:

bin/instance fg

This launches the reference site on port 8080.

Switching between development and production mode

Switching between development and production is as simple as changing a line in the buildout.cfg file.
Other available profiles are commented out.

[buildout]
extends =
    profiles/dev.cfg
#   profiles/prod.cfg
#   profiles/prodtest.cfg

Package management

The most common package management tasks include adding packages, removing packages, pinning the versions of packages. The buildout is configured to read the setup.py file. The big deal here is the settings associated with install_requires. Just look for that in the setup.py file and add the packages that are dependencies. (This is a standard python packaging convention learn more about install_requires).
Here's what it looks like in my buildout.

Additional Background


This is not original, while I've tweaked and hopefully improved things, I lifted the idea of a "profiles" folder from Redturtle https://github.com/RedTurtle/deployments.buildout.plone.
I've seen similar approaches implemented by others, for example Jarn https://github.com/Jarn/buildout calls their profiles folder "cfgs" and Starzel https://github.com/starzel/buildout calls the profiles folder "linkto". I like the name "cfgs" it is most descriptive so I may adopt this naming approach in the future.


October 23, 2016

Gil Forcada: Plone on python 3 status

by gforcada at 2016-10-23T03:53:27Z

2020 is approaching fast, one day at a time, and besides being a nice catchy year, it will be also the year where Python 2.7 will no longer get any security updates.

What that means for Plone? We should hurry up and get a python 3 porting of the whole stack, yes, including Zope, ZTK, ZODB and all the tooling around (zc.buildout, etc etc).

Fortunately quite some tooling and Zope/ZTK/ZODB is already updated and there’s ongoing effort on porting the remaining parts.

The big elephant on the room blocking any porting effort of Plone to python 3 was RestrictedPython, a python distribution that, quoting itself: provides a restricted execution environment for Python.

Note the past on the previous sentence.

Since RestrictedPython is being worked, now it’s high  time for the other python distributions from Plone to be also made compatible with Python 3. Stay tuned for the Plone Conference 2016 sprint report!

Copy&pasting&adapting a set of scripts to track the progress of the porting for the Zope foundation github organization, results here, I made the same but tracking what Plone 5.1 (including our testing environment) looks like on Python 3:

http://jenkins.plone.org/py3/

The code is on collective, so feel free to update the package list.

During the Plone Conference 2016  there is quite some work put on either reducing the amount of dependencies, or updating our stack to use newer (already python 3 compatible) part of the underlying stack.

The clock is ticking and Plonistas all over the world are working hard on it!

October 22, 2016

Maurits van Rees: Ploneconf Sprint Report Saturday

by Maurits van Rees at 2016-10-22T21:33:06Z

    • Updating add-ons for Plone 4.3 and 5. social.like, FacultyStaffDirectory, contentrules mailtogroup, collective.cover.
    • Working on nicer listing of add-ons on Plone.
    • Plone 5 toolbar UI improvements, default icons if they are missing, changed manage portlets sidebar with sensible texts, edit all portlets.
    • plone.restapi. Long discussion about the framing, options listed as url, building basic Angular app, tutorial for search explaining how to setup al kinds of stuff.
    • RestrictedPython to Python 3: lot of work
    • Porting away several packages from ZopeTestCase. Looking for new server for Jenkins nodes. Removed lots of old upgrades from plone.appgrade.
    • Release team: working on signing PyPI uploads.
    • Review Plone documentation, looking how to use Sphinx in more sophisticated way, linking to the source if we mention classes or modules.
    • Documentation on Plone support channels is done.
    • Pyramid. Start using cookiecutter instead of pcreate to create a new project. Tutorials. Working on deform, colander, and demoes of that.
    • Rapido in Mosaic tiles.
    • Video and VR, Plone 5 support for c2.app.streamingaws, created template for 360 degrees viewer
    • Update Zope dependencies, working at fixing some breakage when using newer versions.
    • Resource handling, working on bug with legacy scripts, discussions.
    • eea.facetednavigation, fix issues during upgrade from version 9 to 10, fixed batching issue, working on final blocker for release.
    • Jasonic api for ZODB, working remotely with Jim.
    • plone.app.multilingual, better visibility for selecting translations, inline svg.
    • Make icons great again with inline svg instead of fonts.
    • bobtemplates.plone theming improvements, separate the theming template from plone_addon, add a fat theme template for TTW, added fat theme buildout template, adding Rapido and other stuff, working on wrapper script.
    • Plone marketing, prepared more news items for plone.org, plone.com content changes, marketing ideas for headless CMS, Carol will be interviewing some of you.
    • Plone Cleanup, made it so that you can really get Products.CMFPlone without Archetypes, removed zope.formlib from GenericSetup which was the last one that was using it, zope.globalrequest, reviewing work.

October 21, 2016

Maurits van Rees: Lightning talks Friday

by Maurits van Rees at 2016-10-21T22:04:03Z

Paul Roeland: Plone Open Garden

Sorrento. Lovely spot in Italy. Annual event for past ten years or more: Plone Open Garden. What is it? It is a place that has Plone and Plonistas. It has a hotel as our central place, great food and drink, family friendly atmosphere for open discussions. Nice view. We want to focus on headless CMS. But not only tech, also how this affects our marketing and strategy. People with different skill sets are welcome to join.

When? Not fixed yet, but around 18 till 22 April 2017. Watch for one or more preparatory sprints over the world. Watch for discussion docs and roadmap. Please signal your attention early so we can make a great deal with the hotel.

Eric Bréhault and Philip Bauer: Plone futures

There are different possible futures for us. Valid, possible, and good. Several roads.

  1. CMS. We are targeting that market. It's what we do right now. Lots of plans like moving to Python 3.
  2. Products. Quaive (Intranet), CastleCMS. Targeting specialised audiences. Built on the same technology. Various approaches to UI.
  3. Headless CMS. Different market, like Contentful. Expose Plone as API for Javascript. Compared with comparable solutions, we are way better and we are open source. It is a different market with a lot of potential. We have a management application already in front of it: Plone 5. Unique! Plone 5 is the reference implementation of the UI.
  4. plone.server. All of the above, headless plus Python 3.

Thomas Schorr: Managing revisions in Plone

CMFEditions has been used for a long time for content revisions in Plone. Configured in the control panel. You can view old versions, or revert to them.

Several limitations and issues. History listing and statistics are calculated on the fly, which takes a very long time. If you delete a working copy, old revisions stay in the ZODB, they will not get deleted by anything through the web. Real life example, 50 GB data, out of which 34 GB was in revisions. That may not be common, they were editing several large documents daily.

We created collective.revisionmanager for this customer. Sorted listing of portal_historiesstorage. You can purge revisions or delete entire histories. It maintains a cache for statistics and history data. It has a control panel for the purge policy.

https://pypi.python.org/pypi/collective.revisionmanager

Timo Stollenwerk: Angular2 app

We made an example Angular2 blog app on top of Plone.

https://github.com/collective/tutorial-blog-angular2

Fred van Dijk: From process with love

Talking about processes let's you end up with an empty room. There was a Planning and Organising Sprint in June this year in Amsterdam. Shouldn't we write down our processes? We have between ten and fifteen teams in Plone. Who knows what a PLIP is? Everyone. Who does the roadmap? Framework team? Release team? Roadmap team, is that existing?

How do people who are not here or are introverts, give feedback? How much time does volunteering cost? If we describe tasks, it makes it easier to give a task to someone else.

Release team only does releases to PyPI? What about press releases? News items, tweets, documentation? There is more process here.

Do we need a process team? Yet another team? I will start documenting some.

Hector Velarde: Brazil

Joke in Brazil: Brazil is a country of the future and it always will be. President Lula got lots of people out of poverty. Still big gap between rich and poor. Next president Dilma was impeached. Police used to be nice to protesters, but not anymore.

What has this got to do with Plone? We created a blog add-on, with payment system, to maintain freedom of speech.

David Bain: Gloss

Gloss helps with theming by adding classes. gl-menu, gl-drop, gl-frontend. Diazo makes xslt easy. Gloss is supposed to make Diazo easy.

Other David: Don't get pwnd

Use https! Get a certificate. Nag your sysadmin about it. Free at https://letsencrypt.org Commercial may be better for you.

Don't drop to http if the client tells you.

Only send cookies over https

Get a good score on ssllabs.

Ivan Teoh: Plomino 2.0

Plomino is a flexible and powerful application builder in the Plone UI. Version 2.0 is mainly to support Plone 5. Archetypes support has been removed. Small demo.

https://pypi.python.org/pypi/Products.CMFPlomino

Annette Lewis: Empathetic designer

This is for anyone who needs to give deliverables to other person. Don't let others set your feelings. They will try to bring you down if they see it has an effect. Smile, turn up the corners of your mouth. The person in front of you is inclined to mirror this. Enjoy what you are doing, appreciate what you enjoy. Disassociate from 'toxic' workplaces or persons, you are separate.

Eric Wohnlich: ims.upload

In other solutions we were missing chunked uploads, resuming a failed upload when you retry. The jquery.upload library does support it, so we support it in ims.upload.

Not released or on github yet, I hope to do that soon.

Matthew Wilkes: Saving a start up money

Using Pyramid and Probability. I created a site for a startup that were trying to match cat owners and cat sitters. We needed to enable conversations by phone between the two groups, using Twilio. We made it so it in the end cost far less than otherwise. Lesson learned: don't guess, because we started with a solution that would have cost much more. See http://catinaflat.com

Sally Kleinfeldt: CMFBibliographyAT

This was not Plone 5 ready. We had several options, but decided we may just want to store it outside of Plone. There is http://pleiades.stoa.org: Plone plus Zotero. It would still be a lot of work, but if you are using it, please contact us and we'll see if we can pool resources.

Cris and Sally: Closing words

Thank you to the Microsoft NERD center for hosting us. Thanks Jazkarta and Wildcard for organising. Thanks to MIT Media Lab for provide us the Barton room for the keynote talks, especially Jen. Thanks to the training spaces: District Hall, Landmark Center, ZipCar. Thanks to Gold sponsors cars.com and SixFeetUp, and the sponsors at all the other levels. Thanks to our media partner evenios, especially Armin. Special thanks volunteers Doug Feeney and Michelle Esperanza. Thanks to T. Kim Nguyen for your time, effort, patience, generosity of spirit and just being you. Thanks to our fantastic trainers, our amazing speakers. Thanks to every last one of you who attended the conference.

Paul: "A roaring applause for you two, Cris and Sally!!!!"

Ramon and Victor: Special surprise announcement

16 to 22 October 2017: Plone Conference Barcelona. We will be at the technical university, they support us, they have 400 Plone sites. We want to involve the wider Python community of Barcelona, encouraging others to join, maybe a more general Python track.

Timo Stollenwerk: Sprint kick-off

The sprint starts tomorrow at 9 o' clock, in this room. This is a perfect time for beginners to join and start doing some work. You don't have to be a crack core developer, not at all. You are very welcome, we are very friendly and open people.

Possible sprint topics are on titanpad.