Things Gateway - the Virtual Weather Station Code

◄ Newer Article
◄ Newer Programming Article
◄ Newer Iot Article

Home ⌂

Older Article ►
Older Programming Article ►
Older Iot Article ►

Today, I'm going to talk about the Python 3 code that I used to create the Virtual Weather Station in my previous post .

Nota Bene : The online service that provides weather data for this project no longer exists. You can never depend on a online service, even if you pay for it. The company that runs it (in this case, WeatherUnderground) may get purchased by another company (in this case, IBM) and they go and wreck everything.

The Virtual Weather Station was written using Things Framework, a new communication protocol to connect devices with controllers based on Web technology. The Things Framework consists of a set libraries and modules written in various languages. Each library implements a server that offers the Web Thing API on behalf of the device running the server. The protocol is HTTP, so the server offers a Web interface by embedding a Web Server. That interface contains all the mechanisms to query or control the device and is, therefore, the embodiment of the Web Thing API .

webthing-python

The webthing Python 3 package is an implementation of the Things Framework. When creating the code for a physical or virtual device to speak the Web Thing API, this module supplies the required classes and functions. Like all the webthing libraries in various languages, it creates a server called WebThingServer . It's really a container for two other servers. It employs an instance of tornado , an asynchronous Web server, as well as a zeroconf , a multi-thread hot Multicast DNS discovery service.

The tornado Web server is the window to the wider world for a device. It allows the device to report state to the Things Gateway. The Things Gateway can also manipulate the state of the device through this interface.

The zeroconf server allows the Things Gateway to discover the device's tornado Web server on the local network by announcing its existence via Multicast DNS. This means that you don't have to know the IP address of the things that you have on your network, the Things Gateway can find them on its own.

When writing an application with WoTServer class, a collection of Thing objects are passed to the WoTServer . The WoTServer iterates through the collection and creates the URL-to-handler-method associations required by the tornado Web server. It accomplishes this with the use of the Thing and Property public classes in cooperation with a set of not-as-public helper classes. The Thing and Property classes implement the GET and PUT equivalents that the Web Server needs to implement its services. Therefore, the Thing and Property classes are tightly bound to the Web Server and HTTP.

You may notice that several examples of Things Framework code also refer to Action and Event classes. The Things Gateway doesn’t currently provide a way to use actions or display events, but those are in the works. To simplify my examples, I'm going to omit the use of these classes until they are supported within the Things Gateway.

Writing Code with webthing

The webthing-python code tracks a reference implementation written in Javascript. As such, webthing Python package carries some baggage that is not particularly Pythonic. It's quite functional, but just doesn't quite fit the idioms and style of Python. There are some particular standouts:

Instances of Property are added at initialization time rather than at class load time. This means they are difficult to represent using Python's property descriptor system, which seems like it should be a natural fit.
The "read/write" versus "read only" aspect of a Property is not specified in the Property but, instead, by the presence or absence of a method passed into the constructor of an entirely different class, Value .
The arguments list for the Property constructor have a final catch-all parameter called metadata as a free-form dict instance. Python already has a native method of handling this situation with the kwargs argument.

class ExampleDimmableLight(Thing):
    def __init__(self):
        Thing.__init__('a lamp', 'dimmable light', 'a Web connected lamp')
        self.add_property(
            Property(
                self,
                'level',
                Value(0.0, lambda l: print('New light level is', l)),
                metadata={
                    'type': 'number',
                    'description': 'The level of light from 0-100',
                    'minimum': 0,
                    'maximum': 100,
                }))

(this example is a derivation of the ExampleDimmableLight from the webthing examples )

What would be the ideal interface in Python to represent the concepts of the Things Framework? Creating my vision of an ideal interface was my goal in making my pywot wrapper around webthing . I addressed the issues above and automated some of the boilerplate asynchronous details. The goal was to simplify the coding to this:

class ExampleDimmableLight(WoTThing):
    def __init__(self, config, lamp_hardware):
        super(ExampleDimmableLight, self).__init__(
            config, 'a lamp', 'dimmableLight', 'a Web connected lamp')
        
    level = WoTThing.wot_property(
        name='level',
        initial_value=0,
        description="lamp brightness level",
        value_forwarder=_set_hardware_level,
        minimum=0,
        maximum=100
    )

(See the full pywot ExampleDimmableLight example for more detail)

How pywot works

I started by creating a class WoTThing that derives from the webthing . Thing class. This new class has a fancy class method called wot_property that combines the webthing . Property with a Python descriptor.

class WoTThing(Thing, RequiredConfig):
    @classmethod
    def wot_property(
        kls,
        *,
        name,
        initial_value,
        description,
        value_source_fn=None,
        value_forwarder=None,
        **kwargs
    ):
        kls.wot_property_functions[name] = partial(
            create_wot_property,
            name=name,
            initial_value=initial_value,
            description=description,
            value_source_fn=value_source_fn,
            value_forwarder=value_forwarder,
            **kwargs
        )

(see class WoTThing in the pywot code)

The parameters ( name , initial_value , description , ... ) get saved and used later to instantiate Property objects during initialization time for any derived class. Each time wot_property is called, the method gathers the parameters required to create a Property and squirrels them away in a partial application of a function (see functools.partial ). That partial function is then stored in a class scoped list called wot_property_functions . The partial functions stored in that list will be called later during class initialization time where they will create the actual Property instances.

        if value_source_fn is not None:
            async def a_property_fetching_coroutine(thing_instance):
                while True:
                    try:
                        await value_source_fn(thing_instance)
                    except CancelledError:
                        logging.debug('cancel detected')
                        break
                    except Exception as e:
                        logging.error('loading data fails: %s', e)
                    await sleep(thing_instance.config.seconds_between_polling)

            a_property_fetching_coroutine.property_name = name
            kls.property_fetching_coroutines.append(a_property_fetching_coroutine)

If the Property needs to get values from external hardware or a process (as is the case for any sensor), the wot_property method creates a locally scoped method to poll the underlying controllable device for the value of the Property . The method is a closure over the programmer supplied value_source_fn . This function reads values from that underlying hardware once and need not worry about polling or timing. The server will later start a asynchronous event loop task to do the polling by periodically call the value_source_fn .

            def get_value_fn(thing_instance):
                return thing_instance.properties[name].value.get()
    
            def set_value_fn(thing_instance, new_value):
                thing_instance.properties[name].value.notify_of_external_update(new_value)
    
            return property(get_value_fn, set_value_fn)

Finally, the wot_property method creates a property descriptor using the Python property method. It returns a descriptor for both a setter and a getter for the Property . This sets up the syntactic sugar of allowing the derived class to refer to the value of the Property or set it as if it were directly a member of the class instance.

        current_observation = self.weather_data['current_observation']
        self.temperature = current_observation['temp_f']
        self.barometric_pressure = current_observation['pressure_in']
        self.wind_speed = current_observation['wind_mph']

in comparison to the method using the webthing API alone.

        current_observation = self.weather_data['current_observation']
        self.properties['temperature'].value.notify_of_external_update(
            current_observation['temp_f']
        )
        self.properties['barometric_pressure'].value.notify_of_external_update(
            current_observation['pressure_in']
        )
        self.properties['wind_speed'].value.notify_of_external_update(
            current_observation['wind_mph']
        )

The combination of wot_property with the module scoped create_wot_property method addresses all of the issues from my standout list above. It also handles the the boiler plate code for polling the underlying hardware. This reduces the programmers job to just declaring the Properties of the desired class and providing an asynchronous method to fetch values for the properties. It significantly simplifies the controllable device code, making it easier to both read and write.

The Virtual Weather Station uses three properties:

    temperature = WoTThing.wot_property(
        name='temperature',
        initial_value=0.0,
        description='the temperature in ℉',
        value_source_fn=get_weather_data,
        units='℉'
    )
    barometric_pressure = WoTThing.wot_property(
        name='barometric_pressure',
        initial_value=30.0,
        description='the air pressure in inches',
        units='in'
    )
    wind_speed = WoTThing.wot_property(
        name='wind_speed',
        initial_value=30.0,
        description='the wind speed in mph',
        units='mph'
    )

(see the full code for the Virtual Weather Station in the demo directory for pywot )

Notice that in a webthing version of this code, the description and units would be expressed as members of a dict given to the metadata parameter rather than as direct parameters. When the Property class is eventually instantiated, those two parameters will be put into their proper places within a metadata dict automatically by the create_wot_property partial functions.

    async def get_weather_data(self):
        async with aiohttp.ClientSession() as session:
            async with async_timeout.timeout(config.seconds_for_timeout):
                async with session.get(config.target_url) as response:
                    self.weather_data = json.loads(await response.text())
        current_observation = self.weather_data['current_observation']
        self.temperature = current_observation['temp_f']
        self.barometric_pressure = current_observation['pressure_in']
        self.wind_speed = current_observation['wind_mph']

The Virtual Weather Station action taken by the automatic asynchronous polling loop is the method called get_weather_data . It contains the code to call out to Weather Underground to fetch the current conditions. Once it has its json payload of weather data, it uses standard assignment operators to save the values to the object. Under the covers, though, those assignments are actually performing the actions required by Property and Value objects.

The pywot package also provides a derived class of the webthing WebThingServer class called WoTServer . The only extra functionality it adds is the management of the asynchronous polling methods for the Thing properties. Rather than just use the start/stop methods of the base class, it provides a run method. This allows the class to control the orderly shutdown of the asynchronous polling tasks and exception handling.

To my eyes, the pywot package makes creating a virtual Web Thing pretty simple. Though, as I've said many times, "complexity is in the eye of the beholder". In my next posting, I'm going to talk about more Virtual Web Things that I've created. If you want to jump ahead without me, see the demo directory of my github repo for this project .

Nota bene: pywot is experimental software to prove a concept. I've not uploaded it to create an installable package as it is not stable: as the Things Framework evolves, this module will evolve, too. It has no tests. Use it at your own risk.

twobraids

Search This Blog