=== administrador is now known as giovanoti
=== Rafik_ is now known as Ragik
=== Ragik is now known as Rafik
cr3hi folks! time for a Checkbox tutorial16:02
cr3first, a poll: who here has used Checkbox (previously known as hwtest) in the menu: Administration -> Hardware Testing?16:03
arao/ ;-)16:04
cr3ara and swhcuk: woohoo! thanks for contributing hardware and test information to Launchpad!16:04
danbh_intrepidI have, even posted once on the launchpad page?16:04
cr3danbh_intrepid: thanks, much appreciated!16:05
gueafrt5I have, on hardy :p16:05
cr3Checkbox is a testing framework which makes it possible to run tests and then submit the results somewhere, such as Launchpad for example.16:06
cr3The tests can be automated or manual, they can also integrate third party suites such as autotest from the Linux kernel, LTP from the Linux Testing Project, LSB from the Linux Standard Base, etc.16:06
cr3The tests can be defined very simply in a flat text file. Lets look at one example:16:07
cr3name: audio16:07
cr3plugin: manual16:07
cr3categories: laptop, desktop16:07
cr3requires: alsa.type == 'control'16:07
cr3command: audio_playback $data_path/audio_playback.wav16:07
cr3_description: Testing detected sound card: . $(audio_playback --view) . Did you hear a sound?16:07
cr3The file which contains this test constitutes a suite, within which the name: field above must be unique. it's simply an identifer, no need to worry about it too much.16:08
cr3The plugin: field refers to plugins within the Checkbox framework which provides "manual" and "auto" by default.16:09
cr3The manual plugin will, as its name implies, prompt the user for a question.16:09
cr3The auto plugin will run a test automatically from the command: field described in a moment.16:09
cr3the categories: field limits testing to specific categories of systems which can be: laptop, desktop or server. at some point, maybe thin client will be added16:10
cr3The requires: field is a tricky field which defines whether a test should be run or not. For example, testing Firefox would only really make sense if the package was actually installed. Furthermore, it would not be appropriate behavior for Checkbox to autmoatically install a package just because one of the many tests might need it.16:11
aracr3: and in this case, the alsa.type?16:12
cr3So, the requires field makes it possible to limit the running of a test based on what is called a "registry" which will be described later. Suffice it to say for now, there are registries for packages and devices, so you can limit based on those.16:12
cr3ara: alsa.type is provided by the device registry which maps lshal information. so, try running this: lshal | grep alsa.type. if one of the lines output matches 'control', as defined in the test above, then the audio test will be run16:13
cr3For the Firefox example given above, the requires: field would look like: package.name == 'firefox'16:14
cr3For the advanced user, the requires: field could also provide boolean expressions and lists: package.name == 'firefox' && system.formfactor == 'notebook'16:15
cr3err, make that 'laptop' rather than 'notebook'16:15
cr3So, moving on to the command: field which describes the actual command to run. This field is not absolutely necessary for a manual test however, in this particular case, it happens to be useful in order to play a sound.16:16
cr3As some might notice, the command: field contains a $data_path environment variable defined by the manual and auto plugins which have a data path, which is simply a directory where data files are stored for testing purposes such as audio files in this particular case.16:17
cr3If more environment variables might be needed, these can be defined in another plugin which could be defined in the plugin: field.16:18
cr3Finally, the description: field provides a simple description for the test. In the case of manual tests, this is the actual question asked to the enduser. In the case of automated tests, this is strictly informational.16:19
cr3As some might notice again, the description: field contains variables which might be interpolated before being presented to the enduser.16:19
cr3For those familiar with shell programming, the description is actually interpreted as a here document so more elaborate scripting can be done within the description for ultimate flexibility.16:20
cr3Now that we understand most of the relevant fields for defining a test, these can be defined in a suites test file. For hwtest, these are available under the directory: /usr/share/hwtest/suites16:22
cr3All the files under that directory are automatically parsed for tests, so you can simply drop new tests in that directory16:22
cr3Here is what a minimal automated test would look like:16:23
cr3name: cr316:23
cr3plugin: manual16:23
cr3description: Is cr3 available in #ubuntu-classroom?16:23
cr3That's it, three lines!16:23
cr3However small that might be, three lines could be painful to write for the hundreds if not thousands of tests available in other third party suites16:24
cr3This leads to the concept of plugins which can totally modify the behavior of Checkbox and how it manages tests.16:25
cr3By the way, a plugin is also how Checkbox can be modified to submit test results to another location than Launchpad such as your filesystem for example.16:25
cr3A plugin is simply a Python module which registers for events. A minimal plugin would look like:16:26
cr3class Cr3Plugin(Plugin):16:28
cr3def register(self, manager):16:28
cr3super(Cr3Plugin, self).register(manager)16:28
cr3self._manager.reactor.call_on("report", self.report)16:28
cr3def report(self):16:28
cr3self._manager.reactor.fire("report-cr3", "cr3 is in #ubuntu-classroom")16:28
cr3factory = Cr3Plugin16:29
cr3ok, that output didn't look too good in IRC but it should be simple enough to grock with some explanations16:29
cr3First, the new plugin should inherit from the Plugin class.16:29
cr3Second, it should register to be called on particular events such as the "report" event in this particular example.16:30
cr3The registration is done with this call: self._manager.reactor.call_on("report", self.report)16:30
cr3So, upon the "report" event, the self.report method will be called.16:30
cr3Third, in the report method is defined to perform some action. In this particular case, the event handler is dispatching a new event called "report-cr3"16:31
cr3So, anyone who has registered for this particular event will be called.16:32
cr3A neat trick is that registering for events can be done with regular expressions too! So, the above call could be rewritten to register for all events starting with "report":16:33
cr3self._manager.reactor.call_on("report.*", self.report)16:33
cr3Finally, the plugin should provide a global factory variable which will be the hook for instantiating your plugin.16:34
cr3Then, in a similar way as test suites, you can simply drop your plugin into the corresponding directory and it will be automatically parsed by the Checkbox framework.16:35
cr3The default directory is: /usr/share/hwtest/plugins16:35
cr3Some of the interesting features provided by plugins include:16:35
cr3- Integration with configuration files where some variables can be formalized as required or optional using the required_attributes and optional_attributes class variables.16:37
cr3- Persistence where some information can be preserved seemlessly across invocations of the Checkbox application.16:37
cr3- Access to the registry, which will be described shortly, in order to query information about the system.16:38
cr3That's about it, very few paradigms are meant to be provided  by Plugins in order to lower the barrier to entry as much as possible.16:39
cr3err, that's about it regarding plugins :)16:40
aranow registries, i guess :-)16:41
cr3The registry, which has been mentionned in a couple occasions already, provides a consistent way of accessing hierarchical information in the system.16:41
cr3ara: you read my mind16:41
cr3It is meant to expose a single rooted data structure which can be accessed consistently regardless of where the information might be stored16:42
cr3For example, cpuinfo.flags would access /proc/cpuinfo16:42
cr3Another example, packages.firefox.version would access dpkg output16:43
cr3One of the motivations for provding this kind of data structure is to be able to define the requires: field in tests in a consistent way16:44
cr3Another is to provide plugins with a convenient way to obtain and query information about the system.16:44
araI guess registries require, in general, sudo permissions, don't they?16:44
cr3In fact, a very cool way of querying the registry is by using registry_eval_recursive! This makes it possible to eval an expression across the whole registry16:45
cr3ara: the concept of registries simply accesses information. so, if that information requires sudo privileges, then yes.16:46
cr3ara: however, for the two examples given above using /proc/cpuinfo and dpkg, then simple user privileges are fine16:47
aracr3: thanks for the clarification :-)16:47
cr3A new registry is defined in a similar way as plugins in terms of:16:48
cr31. Deriving a class but, instead of Plugin, the Registry class in this case.16:49
cr32. Creating a factory global variable as a hook.16:49
cr33. Copying the new registry as a Python module under a directory, /usr/share/hwtest/registries in this case.16:49
cr3The main difference is how the Registry class is derived. So, lets look at yet another example:16:50
cr3class Cr3Registry(Registry):16:50
cr3  def items(self):16:50
cr3    return (("foo": 1), ("bar": 2))16:50
cr3factory = Cr3Registry16:50
cr3Then, if I copy this code to a Python module called cr3.py under the registries directory, I could then query the cr3 namespace.16:51
cr3For example, cr3.foo would be 116:51
cr3So, the same concept could be extended to lshal, /proc/cpuinfo, /proc/meminfo, dpkg, lsb_release, etc.16:52
cr3In other words, the new Registry simply needs to define an items method and then the RegistryManager handles the rest.16:53
cr3Another similarity with Plugins is that a repository benefits from integration with configuration files, which will be the final building block to understanding Checkbox.16:54
cr3The configuration file(s) in Checkbox are mostly responsible for defining the behavior of the application.16:55
cr3Checkbox tries to hard code as little policy as possible and delegates this responsibility to the configuration file(s)16:55
cr3These files are in an .ini format and stored in the directory /etc/hwtest.d16:56
cr3The default section of the configuration file, which is typically at the top, defines just a handful of variables:16:56
cr3version: the version of the configuration file. this is stored in the configuration rather than the application because, as mentionned earlier, the configuration is mostly responsible for the behavior of the application16:57
cr3includes: other configuration files to include. for example, the GTK and CLI configuration files which define the interface to use also include the base configuration16:58
cr3plugins: name of the sections in the configuration file(s) which define plugin information, such as the directory where they are stored16:58
cr3registries: similar as plugins, but defines registry information16:58
cr3That's it for the default section. The rest strictly depends on what has been defined above.16:59
cr3For example, if I defined: plugins = cr3/plugins. Then, I would define a section of the same name with plugin related attributes. For example:17:00
cr3directories = /tmp/plugins17:00
cr3persist_filename = /tmp/plugins.persist17:00
cr3blacklist = cr317:00
cr3So, all the plugins under /tmp/plugins will be parsed exacept for the cr3.py plugin because it has been blacklisted.17:01
cr3I'm running out of time so that's about it for the configuration file, the rest could be easily understood from the existing files distributed with the Checkbox (previously known as hwtest) project.17:02
cr3For more information, please visit the project available on Launchpad: https://launchpad.net/checkbox17:02
arathanks for the session!17:03
wasikevincr3: when is the class?17:53
cr3wasikevin: class ended an hour ago, sorry dude17:53
cr3wasikevin: the logs should be available as detailed in the topic: https://wiki.ubuntu.com/UbuntuDeveloperWeek17:54
wasikevincr3: thx17:54
wasikevincr3: sorry, I just noticed it.17:54
cr3wasikevin: if you have any questions, please ask in #ubuntu-testing in case there's another class going on in here17:59
=== rzr is now known as RzR

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!