First off, run either of the following commands from PyCharm’s Terminal window (or the command line, ofcourse).

pip install web.py
# or
easy_install web.py

From PyCharm you can also go to File | Settings, then locate Project Interpreter, for your project. Here you can press the + button, search for web.py then press the Install Package button.

If we take the Getting Started code from the web.py website and place into a .py file

import web

urls = (
    '/(.*)', 'hello'
)
app = web.application(urls, globals())


class hello:
    def GET(self, name):
        if not name:
            name = 'World'
        return 'Hello, ' + name + '!'


if __name__ == "__main__":
    app.run()

Run this and you should see the output http://0.0.0.0:8080. Now, from your preferred web browser type http://localhost:8080. The default result will be “Hello hello!” if you now type http://localhost:8080/PutridParrot you’ll see “Hello PutridParrot!”.

So what’s happening…?

urls = (
    '/(.*)', 'hello'
)
app = web.application(urls, globals())

The web.application creates a web applications with the supplied url’s and the classes that each url maps to. In this instance all url’s map to the hello class. Now the hello class is

class hello:
    def GET(self, name):
        if not name:
            name = 'World'
        return 'Hello, ' + name + '!'

So this class takes all url’s and defines a function for each HTTP method accepts, the case maps to that sent via the browser, hence is uppercase GET (if you make a mistake here you’ll see a Method Not Allowed error from your REST service).

Our GET method takes an argument, in the case of http://localhost:8080/PutridParrot this argument is Putridparrot. If no argument is supplied the code sets name to ‘World’. You get the idea.

Finally we need to run the service application, hence the code

if __name__ == "__main__":
    app.run()

Templating

So this is all well and good, but the output is plain text, what if we are serving web pages, it’d be a lot better to be able to write HTML and embed our Python code in it, like PHP, ASP etc.

web.py includes a templating capability, so if we create a folder for our templates (let’s call it templates) off of our service source. Then add an HTML file, let’s call it index.html, now remove the HTML from this and replace with

$def with(name)

$if name:
   Hello <em>$name</em>
$else:
   Hello <em>World</em>

We’ve got the first line as $def with(name) which basically states that our template is called with the variable name which we then use within the rest of the template. The $ obviously preceeds our Python statements, but other than that it’s probably pretty obvious what’s going on here and fairly standard looking for Python.

Now change our python source to look like this

import web

urls = (
    '/(.*)', 'index'
)

render = web.template.render('templates/')

class index:
    def GET(self, name):
        return render.index(name)


if __name__ == "__main__":
    app = web.application(urls, globals())
    app.run()

We need to tell web.py where our templates reside, hence the line

render = web.template.render('templates/')

Now I’ve changed the class etc. to be our index class, this wasn’t necessary as render.index maps to our HTML file name, but I figured I’d try and make things more like a real world application.

From the command line/terminal run

pip install bottle

Bottle supports a style of defining routes to functions using decorators, for example

from bottle import route, run

@route('/<name>')
def index(name):
    return f"Hello {name}"

run(host='localhost', port=8080)

HTTP Method(s)

By default a @route will respond to a GET HTTP method, but we can change to POST or be more explicit using the following

@route('/<name>', method='GET')
# or
@get('/<name>')
# or POST
@route('/<name>', method='POST')
# or
@post('/<name>')

We can combine the methods using the following

@route('/<name>', method=['GET', 'POST'])

and then within the actual function we can use the request class like this

@route('/<name>', method=['GET', 'POST'])
def index(name):
    if request.method == 'POST':
       return f"POST {name}"
    elif request.method == 'GET':
       return f"GET {name}"

    return "Unexpected"

We can also declare multiple routes to a function like this

@route('/', method=['GET', 'POST'])
@route('/<name>', method=['GET', 'POST'])
def index(name = None):
    if request.method == 'POST':
       return f"POST {name}"
    else request.method == 'GET':
       return f"GET {name}"

    return "Unexpected"

How about some JSON

Obviously JSON is the current “in vogue” format for transferring data. To return JSON instead of a simple string we can use jsonpickle, start off by installing the library using

pip install jsonpickle

We’ll add an example class for our result

class Person:
    def __init__(self):
        self.firstName = "Eddie"
        self.lastName = "Van Halen"

and our code (and additional imports) looks like this

import jsonpickle

@get('/<name>')
def index(name):
    person = Person()
    return jsonpickle.encode(person, unpicklable=False)

The unpicklable=False turns off the pickle additional information to our JSON.

Errors

We can also supply web pages, data etc. based upon HTTP errors, such as 404 errors. Assuming we changed out @get (above) to

@get(‘/name/<name>’)

now URL’s such as http://localhost:8080/PutridParrot will fail to find a matching route and if we supply an error handler for a 404 then we can return some other data, i.e.

@error(404)
def error404(error):
    return "Got a 404!"

References

Bottle: Python Web Framework

In my post Unit testing in Python I covered the basics of writing unit tests within Python.

Testing also sometimes involves writing stubs and/or using mock objects.

As Python is a dynamic language which supports duck typing, we can fairly easily create stub objects, but a mocking framework can make things even simpler. However a good mock framework also gives us the ability to check whether a method was called and assert that it was not called, or was called the correct number of times.

Using MagicMock

MagicMock is a class within the unittest.mock module used to patch methods. Patching being the process of swapping out methods with our mock methods.

Here’s a Calculator class with an add method

class Calculator:
    def add(self, a, b):
        return a + b

Let’s assume we want to mock/patch the add method and ensure it’s called a certain number of times

Note: this is a contrived example as we are explicitly calling the add method, thus know how many times it’s called, but it’s a simple enough example to see how things fit together.

from unittest.mock import MagicMock 

class CalculatorTests(TestCase):

    @classmethod
    def setUpClass(cls):
        cls.c = Calculator()

    def test_add(self):
        self.c.add = MagicMock(name='add')
        self.c.add.return_value = 5

        self.assertEqual(5, self.c.add(3, 2))
        self.c.add.assert_called_once_with(3, 2)

In the above we’re using assertEqual just to demonstrate the use of the return_value. The call to the add method within the assertEqual demonstrates how we still call our patched method in the same way and then finally we use assert_called_once_with(3, 2) to assert that the add method was call once and with the expected parameters.

References

unittest.mock — mock object library

When looking at some built-in functions within builtins.py, I came across the strange syntax

def ascii(*args, **kwargs): 

The *args and **kwargs has a special meaning within Python. The *args will give all function parameters as a tuple and the **kwargs will give a dictionary of key/values.

Note: convention seems to suggest the parameter names args and kwargs, but these are not enforced naming conventions.

Let’s see these two (separately) in action

def show_args(*args):
    for a in args:
        print(a)


def show_key_values(**kwargs):
    for a in kwargs:
        print(a, kwargs[a])


show_args("Hello", "World", 123)
show_key_values(first="Hello", second="World", third=123)

As you can see, in the show_args function we will list each argument out, hence the output to stdout will be

Hello
World
123

The show_key_values expects key/value input, hence the field/property name followed by = and then the value. This will output

first Hello
second World
third 123

In both cases we can also pass no arguments to these methods and nothing will be output, but it’s also not unusual (as can be seen by the ascii function at the start of this post) for the two pieces of syntax to be combined, i.e.

def show_args_key_values(*args, **kwargs):
    for a in args:
        print(a)

    for a in kwargs:
        print(a, kwargs[a])

and now this can be called in the following ways

show_args_key_values("Hello", "World", 123, first="Hello Again")

As you can probably tell from this syntax, you will need to list the list of arguments before the key/value list of arguments, but you can write the following

show_args_key_values("Hello")
show_args_key_values(first="World")

and therefore handle either a list of key/value set of input.

As Python is a dynamic language, unit testing is not only important for verifying code works as expected, but also needs to cover situations that statically typed languages get for free. So a high level of test coverage is useful to ensure event basic methods are working and even named as expected.

Let’s create a simply Calculator class which we will then write unit tests for (this is saved in a file named calc.py)

class Calculator:
    def add(self, a, b):
        return a + b

We can then create a file test_calculator.py and within this we have the following

from unittest import TestCase
from calc import Calculator


class TestCalculator(TestCase):
    def test_add(self):
        c = Calculator()
        self.assertEqual(5, c.add(3, 2))

As you can see, we need to derive our test class from the TestCase class and test methods should use the naming convention test_ at the start of the test method’s name.

Note: the class name does not require the Test prefix/naming convention.

Using PyCharm we need to create a new configuration, from the Python tests configuration section. Select nosetests and then name the configuration, something like Calculator Tests.

Now we can run the tests. If you find nosetest is not installed you can run the Python Console from PyCharm and run the command

pip install nose

setUp/tearDown

Like most unit testing frameworks, the TestCase class can use a setUp and/or tearDown method for setting up the test case context and cleaning up after each test method is run.

For example

class CalculatorTests(TestCase):

    def setUp(self):
        self.c = Calculator()

    def test_add(self):
        self.assertEqual(5, self.c.add(3, 2))

    def test_subtract(self):
        self.assertEqual(2, self.c.subtract(7, 5))

setUpClass/tearDownClass

TestCase also includes setUpClass and tearDownClass for setting context for all test methods within a TestCase.

For example, here’s the previous TestCase but using the setUpClass to create the calculator for all test methods

class CalculatorTests(TestCase):

    @classmethod
    def setUpClass(cls):
        cls.c = Calculator()

    def test_add(self):
        self.assertEqual(5, self.c.add(3, 2))

    def test_subtract(self):
        self.assertEqual(2, self.c.subtract(7, 5))

Note: the @classmethod is required in this example code or the setUpClass is not treated as a class method as is expected.

References

unittest — Unit testing framework
Creating and running a Python unit test

Python includes a bunch of built-in exception classes and we can easily create our own by creating a class which derives from Exception.

For example, here’s a MethodNotImplementedException type

class MethodNotImplementedException(Exception):
    def __init__(self, *args, **kwargs):
        pass

    def __str__(self):
        return "Method not implemented"

In this example I’ve overridden the __str__ method to return a simple string when we print the exception to stdout.

We need not create specific exception types but obviously this allows us to also catch specific exception types.

Here’s an example of some code that raises (throws) an exception and the code to catch and handle an exception

class Animal:
    def name(self):
        raise MethodNotImplementedException()


a = Animal()
try:
    a.name()
except MethodNotImplementedException as e:
    print(e)
except Exception as e:
    raise
finally:
    print("finally called")

We create a try block and exceptions are caught using the except statement. As can be seen we can catch and filter each exception type we want to handle and include a finally block as required.

To rethrow and exception we simple call the raise method.

In my previous post I looked at some of the basics of Python. In this post I want to dig further into the syntax etc. for classes within Python.

Class naming conventions

We prefix a class name with the class keyword. Python naming convention suggests the class name should be Pascal case, i.e. ClassName.

The method naming convention, by default, should be all lowercase with underscores (snake case) to separate words, i.e. method_name.

Private/protected methods and instance variables should have an underscore prefix, i.e. _private_name.

Defining a class

The special method name __init__ can be thought of a little like a constructor, however we cannot overload it, but we can pass arguments into it

class Animal:
    def __init__(self, name):
        self.name = name

Note: Whilst it might appear possible to have multiple __init__ methods, it’s the last one declared which seems to be the one available to calling code.

We terminate the line that the class keyword is on with the colon and then methods or variables start on a new line and indented with a tab.

The self parameter passed to __init__ or any other class method is passed implicitly, i.e. we do not supply the self, Python does that for us.

Inheritance

Python classes support inheritance (even multiple inheritance) by declaring the base classes in a comma separated list within parenthesis, i.e.

class Animal:
    def __init__(self, name):
        self.name = name


class Mammal:
    pass


class Dog(Animal, Mammal):
    def __init__(self):
        super().__init__("Dog")

In this example you can see how we can call the base class’ __init__ method. The Dog class derives from both Animal and Mammal classes (in this instance the Mammal class has no methods etc. hence uses pass statement to create the empty class).

Calling base methods in Multiple Inheritance

Multiple inheritance always has the issue around what base method to call in situations where there’s multiple methods of the same name, for example if we have the following

class A:
    def name(self):
        print("A")

class B:
    def name(self):
        print("B")


class C(A, B):
    pass

c = C()
c.name()

then what base class method is actually called via c.name().

A quick note: In the above as we’re missing the method name() on the class C, this can be seen as the following

class C(A, B):
    def name(self):
        super(C, self).name()

The order of resolving the methods to be called is the Method Resolution Order (MRO).

For our simple example, the first name() method located depends upon the order of inheritance, i.e. in the example above A’s name() is called, switching the code for C to

class C(B, A):
    pass

results in B’s name method being called.

To ensure we are explicit about which code is called, it’s best to override the name method in the subclass like this

class C(A, B):
    def name(self):
        B.name(self)

and obviously this ensures B’s name is called.

Methods

We declare methods within a class using the keyword def and all methods are virtual, i.e. can be overridden in a subclass. Here’s a basic example over method overriding

class Animal:
    def name(self):
        pass


class Dog(Animal):
    def name(self):
        return "Dog"

In this example we’re using the Animal a little like an abstract class and then implementing the name method in derived classes.

Member variables/fields

Let’s rewrite the previous code to now use a member variable in the base class to store the name of the Animal.

class Animal:
    _name = ""

    def name(self):
        return self._name


class Dog(Animal):
    def __init__(self):
        self._name = "Dog"

In this case the prefixed _ denotes a protected/private variable. In most OO languages this encapsulates the member variable in such a way as to ensure it’s no accessible outside of the base and derived classes, however in Python it’s still available, and thus this will return the value stored within the _name variable. In PyCharm we get a hint that we’re accessing a protected member, but ultimately it’s still accessible, as per the example below

a = Dog()
print(a._name)

Empty classes

In some cases we might want to create an empty class type, such as

class Duck:
    pass

If we need to then create the equivalent of a struct at runtime, then we can simply dynamically “add” variables like this

d = Duck()
d.name = "Duck"

this leads us onto duck typing…

Duck typing

Duck typing allows us to declare Python type for use in other types that simple expect certain methods/variables to exist, for example

class Car:
    def __init__(self, engine):
        self.engine = engine

    def engine_size(self):
        return self.engine.size


class EconomicalEngine:
    size = 1.0


class SportEngine:
    size = 3.5


car = Car(SportEngine())
print(car.engine_size())

In this example our Car has an engine but we do not define an Engine type, instead we simply state that the Car expects a type with a size variable. This does ofcourse means we could pass in anything with a size variable (which ofcourse might not always make much sense).

Static/class methods

Up until now we’ve seen instance methods on classes. Python also includes two decorators that can be used in conjunction with our methods to make them acts in a similar way to static methods.

The first of these is the decorator @staticmethod, for example

class Dog:
    @staticmethod
    def animal_type():
        return "Dog";

# and we call the method like this
print(Dog.animal_type())
# or
d = Dog()
print(d.animal_type())

Python also includes another static-like method type which is decorated with @classmethod. A class method differs from a static method in that it includes an implicit cls variable (by convention cls is used for the first argument). For example

class Dog:
    @classmethod
    def name(cls):
        return "English Springer Spaniel"

# and we call the method like this
print(Dog.name())
# or
d = Dog()
print(d.name())

So what’s the difference? Both appear to allow us to work in a “static” method way but the @classmethod allows us to still get an instance of the class, but it appears that this is almost like passing an instance of a new class into the method, i.e. these appear to be equivalent. Let’s assume we have this

class Dog:
    _name = "Dog"

    @classmethod
    def name(cls):
        print(cls)
        return cls._name

d = Dog()
d._name = "English Springer Spaniel"

print(d.name)
# same as
print(Dog().name)

In the above we change the instance variable _name but both print statements will output “Dog” so appear functionally equivalent.

I’ve used Python (or more specifically IronPython) in the past to use Python as a scripting engine for my app. but never really bothered with Python beyond that. However, I decided I need to spend a little more time with Python, so am going to post a few “getting started” posts, starting with this one…

Note: This post any others on Python are aimed at developers coming from another language, such as C#, Java or C++. So I will not be going over every capability of the language but will aim to cover those main areas of syntax etc. to simply get things working in Python.

Python basics

Python files are usually saved with the .py extension and is interpreted, although there are apps for compiling the scripts to executables.

Python is a dynamic language, hence we do not need to declare the type of a variable.

Python uses whitespace and indentation to denote blocks of code, i.e. in C#, Java etc. we’d use curly braces { } to denote a block of code whereas Python expects the code blocks to be tabbed, for example in C# we might have

public int Pi()
{
   return 3.14;
}

the same function in Python would be

def pi():
    return 3.14

This also demonstrates creating a function in Python (see Functions, below for more information on function definitions).

Also notice that Python does not use line termination like the semi-colon in C#. Java etc. This means if we need to extend our code onto another line we can use the backslash, i.e.

def pi():
    return \
    3.14

Types

Python is a dynamically typed language which has good and bad points. This basically means when we declare a variable name it’s automatically created and it’s type is automatically declared based upon it’s usage. The type can change as we assign different types to the variable, for example

value = "Hello World"
# value is of type str
value = 123
# value is now of type int

Note: # is used for a single line comment, we can use “”” to start and end a block of code to use as multi-line comments, although technically speaking these denote multi-line strings.

Python types include the usual types, such as a string (str), int, long, float, boolean along with complex types, tuples and more.

If we need to find the type at runtime then we can use the type method, for example

value = (1, "Hello")
print(type(value))

In this code, the first line creates a tuple and the next line outputs (to the console) the type of the value variable, in this case the output will be

<class 'tuple'>

Enumerations

Enumerations are declared via the Enum type, we need to import the enum module and then we can declare our Enum subclassed type as follows

import enum

class Gender(enum.Enum):
    Male = 1,
    Female = 2

# in use
g = Gender.Female

Functions

Python allows us to define function using the def keyword, here the example we introduced earlier

def pi():
    return 3.14

def is used to declare or define our function, arguments may be passed within the parenthesis, in this instance no arguments exist. Finally we end the function declaration using the colon. The actual statements that are executed within the function are then below the declaration and the code is indented.

Executing functions uses pretty standard syntax, i.e.

value = pi()

Although Python does a good job of type inference when we have code like the following

def add(a, b):
    return a + b

we have a situation where both of the following bits of code work

strValue = add("2", "3")
intValue = add(2, 3)

but in the first instance we get the string “23” and in the second we get the int 5.

Obviously if our intent of that the add function it for numerical types then we will need to give type hints, for example

def add(a: float, b: float) -> float:
    return a + b

The strange thing here (if you’re used to similar techniques in F#, for example) is that running the previous example of the add function with strings and ints will still work, i.e. still returns “23” and 5. So it’s debatable how useful hints are. Certainly PyCharm will display a “hint” over the string arguments and tell us the expected type, but ultimately we can still pass in a different types for the add function arguments.

If you want to include type hints but not show them in your source or if you want to hint on existing code that you do not (or cannot) edit. Then you can include a .pyi file along with the .py file including the type/function etc. and separate the hints from the actual implementation. So for example our .py file might have

def add(a, b):
    return a + b

and the file with the same name but .pyi extension would have

def add(a: float, b: float) -> float: ...

Object Orientated

Python allows us to define classes uses the class keyword, for example

import http.client

class WebClient:
        def __init__(self):
            self.url = "www.google.co.uk"
            self.method = "GET"

        def submit(self):
            connection = http.client.HTTPConnection(self.url)
            connection.request(self.method, "/")
            response = connection.getresponse()
            data = response.read()
            print(response.status, response.reason)
            print(data)

client = WebClient()

The __init__ function is the equivalent of a constructor and self is analogous to this in C#. Within the __init__ we have declared new variables along with some default values, i.e. the url and method variables.

In the case where we do not have anything to setup/initialize in the __init__ function we can use the pass keyword. Here’s an example of this along with how we can derived/subclass a type

class Animal:
    def __init__(self):
        pass

class Dog(Animal):
    def __init__(self):
        self.legs = 4

A good old, OO example whereby Animal is the base class and Dog subclasses this and adds functionality/fields etc.

As you’ve probably already noticed, Python doesn’t include a new keyword to create an instance of a class we simply use

animal = Dog()

Importing modules & packages

Like most languages, we can create reusable source code modules (and packages) that can then be imported into our source code, for example let’s import a Python standard library for connecting to an HTTP endpoint

import http.client

connection = http.client.HTTPConnection("www.google.co.uk")
connection.request("GET", "/")
response = connection.getresponse()
data = response.read()

In the code above, we import the http.client library and then create an HTTPConnection instance which we then use to connect to google.co.uk.

We can import multiple modules using comma separated values, i.e.

import http.client, sys

Creating modules & packages

Modules are simply ways to define reusable code within separate files. Packages are is essence namespaces which can contain multiple packages and modules.

To create a module, we simply create a file, for example web.py and include our functions and types etc. within that file, so web.py looks like this

import http.client

class WebClient:
        def __init__(self):
            self.url = "www.google.co.uk"
            self.method = "GET"

        def submit(self):
            connection = http.client.HTTPConnection(self.url)
            connection.request(self.method, "/")
            response = connection.getresponse()
            data = response.read()
            print(response.status, response.reason)
            print(data)

and it our file that uses this module, we have the following

import web

client = web.WebClient()
client.submit()

Packages now take this a little further. Still using the file based approach. Packages are simply directories with one or more module file, but every package must include a file named __init__.py which can be empty. This __init__.py file may contain an __all__ variable which defines what modules are exported, i.e. non-exported modules can be internal or private to the package or exported and visible to those using the package. For example our __init__.py file might look like this

__all__ = ['WebClient']

# example within multiple exports
__all__ = ['WebClient', 'SomethingElse']

However the above only seems to be used when using the alternate import syntax, for example

from web import *

See the answers to Can someone explain __all__ in Python? for much more information on the use of __all__.