SpinachHeadanyone in Cali?01:56
pleia2SpinachHead: might want to try #ubuntu-california01:57
SpinachHeadjust trying to figure out what 1600 UTC time is here.  I am stuck at work on a win computer and can't use terminal01:58
ubottuCurrent time in Etc/UTC: January 27 2010, 01:58:5601:58
cjohnstoni tihkn01:59
SpinachHeadokay, so the classes haven't started?01:59
cjohnstonThey started Monday..01:59
cjohnstonEach day for 5 hours01:59
cjohnstonFor the US, todays classes are already over02:00
SpinachHeadokay... I wanted the Ubuntu One class guess I missed02:00
macothat was like 7 hours ago02:00
nigel_nbSpinachHead, https://wiki.ubuntu.com/UbuntuDeveloperWeek see logs here02:01
ardchoillenigel_nb: Thanks02:01
nigel_nbnp :)02:01
ardchoilleIs there a "wish list" or some other way to request a certain type of class?02:04
cjohnstonardchoille: what are you looking for?02:05
ardchoilleFor instance, I'd like to learn how to write apps in python-gtk and use glade to design the ui and have the glade file usable in the app. I've tried online tutorials but can never understand them02:05
ardchoilleIt's be nice to have a class that starts with "this is a glade file and this is how you call the file componenets in a pygtk app: or something similar02:06
ardchoilleJust anidea for a class02:06
cjohnstonardchoille: have you checked out the past classes to see if anything similar has been taught?02:06
cjohnstonthat would help in finding someone who could teach it02:06
ardchoilleI don't recall anything like that but I should check02:06
ardchoilleIs there a list of past classes?02:07
cjohnstonin the topic there is a link to the Classroom page...02:07
cjohnstonfrom the classroom page there is an archive area02:07
nigel_nbardchoille, you can check MOTU school first https://wiki.ubuntu.com/MOTU/School and previous dev week session https://wiki.ubuntu.com/UbuntuDeveloperWeek/Previous02:08
ardchoilleThe problem is that all of the "getting started with ubuntu development" classes I find always deal with packaging an app and that's it. I never see a "how to write an app in pygtk" type of thing. While packaging is a great skill to have, it's also good to know how to write an app02:11
macobut thats not what ubuntu development is...02:11
macothat'd be upstream development02:12
ardchoilleAh, ok, I misunderstood the meaning of ubuntu development"02:12
macomany ubuntu developers dont know how to program beyond maybe a simple bash script to run at the end of package installation02:12
ardchoilleI did not know that02:13
macoyou might be more interested in ... i think its called gnomelove02:13
macoi think thats the name of their wannabe-gnome-dev mentoring stuff02:14
macokde has "junior jobs" which are simple bugs for newcomers, but i dont know of any mentoring for "hi i want to learn qt"02:14
macoand then there's kernelnewbies for people who want to learn to hack the kernel02:14
ardchoillemaco: Thank you for the information02:15
ardchoilleI always thought the packagers were programming guru's too02:17
macoi know one guy who does no programming and another that knew cobol 25 years ago but didnt learn a modern language til after becoming an ubuntu dev02:18
ardchoillemaco: Thank you!02:30
ardchoillemaco: http://live.gnome.org/GnomeLove is an excellent starting point for what I want to do :)02:30
ardchoilleHmm.. I wonder if I get good at this, I might be able to teach a class in here at some point in the future02:32
cjohnstonardchoille: sounds good to me02:32
macoi need to learn where kde keeps equivalent docs. been a kde user for a year, and i still only know how to do gnome programming02:33
linuxmansomeone speek portuguese?03:24
cjohnstonhggdh: i think if hes around03:25
hggdhI am here03:35
cjohnstondo you speak portuguese?03:35
hggdhcjohnston: yes, I do03:36
hggdhlinuxman: why you need Portuguese?03:36
linuxmanyes  speek03:36
linuxmanvc so falam ingles03:36
hggdhlinuxman: PVT03:37
hggdhlinuxman: a liŋua oficial por cá é o ɪŋles03:37
cyborg_7no hacking i are noobs03:58
=== jamalta is now known as jamalta-afk
=== Yos_ is now known as Yos
=== nixternal_ is now known as nixternal
=== SmrtySsa is now known as SmartSsa
=== jarlen_ is now known as jarlen
bittinhttp://fc08.deviantart.net/fs71/f/2010/027/7/1/eeedebian_wmii_by_bittin.png me likes <313:20
=== jamalta-afk is now known as jamalta
=== breiner is now known as breinera
=== cjohnston is now known as FFEMTcJ
=== FFEMTcJ is now known as cjohnston
dholbachIf you are a bit confused, have a look at - it will explain, why you are here: https://wiki.ubuntu.com/UbuntuDeveloperWeek15:59
dholbachOn an organisational note: if you have questions, please ask in #ubuntu-classroom-chat and prefix them with QUESTION: so they stand out.16:00
dholbachAlso, if you have questions but are afraid to ask in English head to one of the channels mentioned at the bottom of https://wiki.ubuntu.com/UbuntuDeveloperWeek16:00
dholbachFirst up is the fantastic John Arbash Meinel - he'll talk about "Writing good test-cases"!16:00
dholbachjam: the floor is yours!16:00
jamgreetings all!16:01
jamI'm happy to see so many names on the channel.... hopefully you all aren't just lurking16:01
jamwaiting for the next speaker :)16:01
jamI'm going to be pretty loose with the discussion16:01
jamso feel free to ask questions16:01
jamcjohnston has graciously offered to help me keep track16:02
jamSo what is a 'good' test case16:02
jamGenerally speaking, a test case is meant to verify that something is happening correctly.16:03
jamOr looking at it differently, when something *isn't* happening correctly, the test case should fail.16:04
jamHowever, there are still a lot of subtle aspects.16:04
jamI would say that the desirable features of a test case break down into16:04
jam1) Sensitivity (the likelyhood that the test case will break when something goes wrong)16:04
jam2) Specificity (the likelyhood that the test case will break and what you are testing has not gone wrong)16:05
jam3) Performance16:05
jamIt is easy to focus on say 1, but to have a *good* test, all 3 are important16:05
jamIt is great to have a huge test suite that covers every possible aspect of your code, and all permutations16:06
jambut if it takes 3 weeks to run, it is often not very useful for development16:06
jamLikewise, a test with low specificity will be breaking all the time16:06
jamand won't really help you isolate the problem16:06
jamYou can argue that 4) coverage is a property, but I would argue that it is a desirable property of the whole test suite, and not a great property of a single test16:07
jamPersonally, I think there are a range of 'good' tests, generally dependent on what aspect you are focusing on16:10
jamI personally think that having lots of focused tests is better than having a single 'test' that is testing lots of aspects16:10
jambut integration tests are still useful an needed16:11
jamSo how about we put together an example16:12
jamI'll use python, since that is my preferred language16:13
jamand it has a decent unittest suite16:13
cjohnstonQUESTION: Even after weeks of testing by most experience testers, very robust apps breakdown all of a sudden, what might be the reasons behind it?16:13
jamANSWER: I think that complexity is generally a major factor causing 'bugs' in software.16:14
jamIt is often considered inherent in any sufficiently developed program.16:15
cjohnston< Omar871> QUESTION: according to what I learned in college, a 'good' test is that one that makes that system break/crash, to show where the problem is, how true could that be?16:15
jamGenerally, this means there will be some sort of permutation of objects which has not been tested16:15
jamA goal of development is to manage complexity (generally by defining apis, separation of concerns, etc)16:16
jamgood software can then have good tests that test a subset, without *having* to manage the permutation problem16:16
jam(but generally, abstractions can 'leak', and ... boom)16:16
jamANSWER <omar>: I think I understand you to mean the "inverse"16:17
jamwhich is that a good test is one that provokes a problem in the system16:17
jamI think that for a regression style test, it is certainly important that the test would trigger the bug that you are trying to fix16:18
jamHowever, once a bug is fixed, you would certainly expect it to not provoke the problem anymore16:18
jamSo it is certainly important that a test exposes a weakness16:19
jamI guess put another way...16:19
jamIf I had a bug, and wrote a test case, and it *didn't* trigger the bug, that test doesn't have a lot of worth (for fixing that bug)16:19
jam(Which often accidentally happens if you fix the bug before writing the test)16:20
jamn3rd: We were just discussing in our group about "coding creates bugs... are we reducing the number of bugs faster than we are creating them?"16:22
jamn3rd also brings up a decent point16:24
jamusers often find bugs that developers don't think of16:24
jamoften because they use software in ways that weren't anticipated16:24
jamoften this goes back to the permutation issue16:25
jamit isn't possible to test every possible permutation16:25
jam(well, rarely possible)16:25
cjohnston< n3rd> jam, so the users are passive tester?16:26
jamA file with just 20 bytes has 8^20 = ~10^48 permutations16:26
jamWell, I would say that users are often pretty *active* testers16:27
jamhowever, they don't make good automated test suites16:27
jamI suppose that would be my:16:27
jam4) Reproducible16:27
jam(the chance that running the same thing now will give you the same thing it gave you before)16:28
jamIt is somewhat related to specificity16:28
jamAs an unreproducible test has low specificity16:28
jam(it breaks for reasons that you aren't trying to test)16:28
jamI guess I should also mention... if a user used every intermediate version of my program, they'd run into a lot more bugs16:29
jamAs a developer I fix a huge amount of things before it gets released16:29
jamit is just that often the set of *remaining* bugs16:29
jamare ones that I had not anticipated very well16:30
jamAnyway, I think it is useful to remember what the strengths of a given style of testing are.16:34
jamYou can have automated tests (unit and integration tests), manual (interactive) testing, foisting off the software on your users16:34
jamI do think that testing at each level is useful16:34
jamand trying to test things at a the wrong level introduces more pain than benefit16:35
jamHaving an absolutely bulletproof piece of software that doesn't do what your users want isn't particularly beneficial16:35
jamSo having user testing in your feedback loop is certainly a good thing16:36
jamHowever, giving them a buggy PoS is also not going to make them very happy16:36
jamI'm certainly a fan of multi-tier testing, including automated testing16:36
jamhaving a small fast test suite that is likely to expose bugs is nice on 'must pass before making it to trunk'16:36
jamhaving a slower but more thorough "must pass before releasing to users"16:37
jamand for some systems adding a "must be tested by a human interaction" can be inserted in there as well16:37
jamIf the first one takes much more than 5 minutes, it often causes grief when trying to get development done16:38
jambut the second can take a day, and still not slow down your release cycle16:38
cjohnston< Omar871> QUESTION: Could the efficiency and effectivenes of the testing process depend on the licensing type of the software we are making?16:44
jamANSWER: I don't think it would change dramatically16:45
jamIf the license is open, it does allow users to do introspective testing which would just not be possible otherwise16:46
jamhowever, few users can really develop your software anyway, so it certainly shouldn't be relied upon as a source of improving correctness16:46
jameven if your users are very intelligent they almost always16:46
jam1) don't know your code16:46
jam2) don't have time to be doing your work for you :)16:46
jamI think Linus gets a bit of a boost, because there are lots of developers on the project, not just users16:47
jamCertainly a "lot of eyeballs" requires eyeballs that can see the code16:48
jamand with enough, you can have a person to specialize for any given subset, which also helps in defect localization (hence 'shallow')16:49
cjohnston < hggdh> QUESTION: are there any considerations for *usability* testing (this is one thing that users would certainly be able to perform)?16:50
jamANSWER: I think that usability testing is certainly important16:51
jam(10:35:23 AM) jam: Having an absolutely bulletproof piece of software that doesn't do what your users want isn't particularly beneficial16:51
jamThere is an argument whether this specifically falls under the standard category of 'testing'16:51
jam(market research is certainly important to developing software, but it isn't "testing" :)16:52
cjohnston< strycore89> QUESTION : how is testing of graphical applications done ? (For example PyGTK apps)16:56
jamIME, you can test PyGTK (and PyQt) without actually showing dialogs16:56
jamboth of them support updating widgets by programattically setting values16:57
jamand then triggering events16:57
jamIn that case, they can be tested in the same fashion as any other unit testing16:57
jamhowever, it doesn't test the visual representation16:57
jamwhich is a valid level to test16:57
jamThere are also gui testing suites that can be used16:58
jamI forget the exact name of one (suikili?)16:59
jamWhich uses screenshots16:59
jamand some work for marking the regions you care about16:59
dholbachthanks jam for giving this great talk!17:02
* dholbach hugs jam17:02
dholbachnext up are henninge and adiroiban, the dynamic duo that'll explain "Launchpad Translations under the hood" to you!17:02
henningeHi everybody, thanks for coming!17:03
henningeToday we want to help you understand the inner workings of the Launchpad Translations application (Rosetta) and take you for a walk through the source code. We hope that this will enable you to scratch your own itches you have about Rosetta and to contribute to its source.17:03
henningeSo the intendend audience is17:05
henninge    * Developers wanting to contribute to Launchpad Translations but are not yet familiar with the internal structure of the application.17:05
henninge    * Interested maintainers of translations in Launchpad and translators that want to have a better understanding of how and why Launchpad Translations does what it does.17:05
henningeWe hope that by the end of the session you have a good overview of17:05
henninge# how translation data is stored in LP translation (db schema)17:05
henninge# how the source code is organized17:05
henninge# what to expect when diving into the source code17:05
henninge# where to start when trying to hack on Launchpad Translations.17:05
henningeIt is not the goal of this session to introduce you to Launchpad development in general. That will be covered in a different session by Karl Fogel.17:06
henningeWe encourage you to visit that on Friday.17:06
henningeThursday, sorry17:06
henningeAnyway, we are happy to take questions. We will keep the session open towards the end to see what questions come up and what your interests are.17:07
henningeWe will be taking questions in intervalls, so cjohnston will nicely hold them until we ask for them. Thanks.17:08
henningeGo, adiroiban ;-)17:08
adiroibanI will continue with Gettext Basics17:08
adiroibanYou need to understand how gettext is used to internationalize computer software. You should be familiar gettext documenation but we will give you a short run-through of those parts that are important for Rosetta.17:08
adiroiban= PO files =17:09
adiroibanGettext stores translations in so-called portable object files, abreviated as PO files. They contain data sets of msgid and msgstr, the former containing the English original string, the later containing the translations of that string. They may be prepended by special comments that convey information about the string that is being translated, like in which source file it was found. Here is an example:17:09
adiroiban#: src/coding.c:12317:09
adiroibanmsgid "Thank you"17:09
adiroibanmsgstr "Merci"17:09
adiroibanGettext states that msgid could be anything to indentify the string in the source code and not necessarily the English original string. Using the full English original string as the msgid, though, has proven to be the most convenient way to work on translations and is _the only form_ that is fully supported by Rosetta.17:10
adiroibanThe first msgid in a PO file is empty and its msgstr contains meta information about the file. The minimum information here is the MIME Content-type of the file but usually a lot of other information is included, too.17:10
adiroibanHere it is17:11
adiroibanmmsgid ""17:11
adiroibanmsgstr ""17:11
adiroiban"Project-Id-Version: PACKAGE VERSION\n"17:11
adiroiban"Report-Msgid-Bugs-To: \n"17:11
adiroiban"POT-Creation-Date: 2009-01-26 12:28+0000\n"17:11
adiroiban"PO-Revision-Date: 2009-01-26 12:28+0000\n"17:11
adiroiban"Last-Translator: Foo Bar <foo.bar@canonical.com>\n"17:11
adiroiban"Language-Team: French <fr@li.org>\n"17:11
adiroiban"MIME-Version: 1.0\n"17:11
adiroiban"Content-Type: text/plain; charset=UTF-8\n"17:11
adiroiban"Content-Transfer-Encoding: 8bit\n"17:11
adiroibanThe standard naming convention for PO files is to use the language code, so in this case fr.po.17:11
adiroiban= Translation templates =17:11
henningeQuestions so far?17:12
cjohnstonhenninge / adiroiban you aren't using slides are you?17:12
cjohnston< Navaneeth> QUESTION: Is gettext portable to other OS?17:13
adiroibanno slides17:13
adiroibanthe content is on the wiki17:13
henningeNavaneeth: The PO format is machine independent.17:14
adiroibanGettext is available on many operating systems17:15
adiroibanI don't think you need to worry for gettext portability17:15
adiroibanI will continue with PO Templates17:15
adiroibanWhen translatable strings are extracted from source code using xgettext or intltool, they are put into a file which is commonly referred to as the translation template. Its format is identical to that of a PO file but all the msgstr lines do not contain any translations. These files are intended to be used for creating new PO files, so they also contain the header information but with most fields left with empty or generic values.17:15
adiroibanSince a PO template is not really a separate file format it does not find much mention in the gettext documentation. Also, because its content can be generated from source any time (like during a build), most projects don't include it in their repository. Only PO files contain valuable information for a project, the translations themselves, and are therefore included in the source code repository.17:16
adiroibanThe standard naming convention for PO templates is to use the translation domain with the extension .pot, for example myproject.pot.17:17
adiroibanHow translations are done in the simple gettext context:17:18
adiroiban1) To start a translation into a new language for a project, the following steps are necessary:17:18
adiroiban* Either the project maintainer or the translator creates a template from source code.17:19
adiroiban* The translator fills out the template with the translations for each msgid.17:19
adiroiban* The translator saves the file in the source tree as languagecode.po (lsee above), ususally in a directory called po.17:19
adiroiban* The translator or somebody with commit rights commits the file to the repository.17:19
adiroiban* Whenever the package is built, the translations are processed so that they are available at run-time (out of scope here).17:19
adiroiban2) To change translations, the steps are simpler:17:20
adiroibanSorry. Launchpad Workflow :)17:20
adiroibanWhen using Launchpad to translate a project, the steps are slightly different because the PO files are kept in Launchpad for the translators to work with. From Launchpad they are mirrored into the source tree to be used at build time.17:20
adiroiban* The project maintainer uploads the PO template file to Launchpad.17:21
adiroiban* Translators go to Launchpad to translate the English strings that now appear in the web interface.17:21
adiroiban* The project maintainer downloads all PO files whenever they want to, usually to prepare a release of the software.17:21
adiroiban* Nowadays the upload and download should happen automatically from and to Bazaar branches in Launchpad so that the maintainer always has a mirror of the latest translations in the branch, while changes to the PO template are automatically propagated to Launchpad. The next step will be automatic generation of PO templates from the source code in a Bazaar branch.17:21
henningecjohnston: Questions? ;)17:22
cjohnstonI don't see any17:22
cjohnstonone just came17:22
cjohnston < Navaneeth> QUESTION: PO files need to deployed along with the application?17:22
henningecjohnston: actually, they are converted to a binary format first "MO".17:23
adiroibanYou will deploy the MO files (machine object)17:23
henningeNavaneeth: ^17:23
henningeNext, we'll go into how this structure is mapped into the Launchpad database.17:24
henningejtv drew up this amazing diagram:17:24
cjohnston< Omar871> QUESTION: What's a PO file?17:25
henningeOmar871: the actual translations are stored in PO files. It contains English strings and their translation.17:25
henningeThat's what translators work with when translating.17:26
henningeBack to database schema.17:26
henningeYou can see the main tables used for Launchpad Translations in the digram. PO templates are mapped into the database using four tables.17:26
henningePOMsgID is a look-up table for all the English strings that are being translated.17:27
henningeSo it gives each string a numeric ID.17:27
henningePOTMsgSet holds all the data related to an original English string as found in a PO template, one database entry per msgid entry in the file. It refers to the actual English strings only by their IDs in POMsgID. This represent one paragraph/entry from a PO template (msgid, msgid_plural, context, comments).17:27
henningePOTemplate holds the meta data related to a PO template as it has been imported, most notably the original path name, the translation domain, the original header and a flag if this template is active or not.17:28
henningeTranslationTemplateItem is a linking table because of the n:m realtionship between POTMsgSet and POTemplate which message sharing introduces. Not only does a PO template file contain multiple msgid entries, the same msgid may also appear in multiple PO template files, if the same template is used accross different series of a a project.17:28
henningecjohnston: ^17:28
cjohnston< n3rd> QUESTION: Can we not give auto suggestions/transitions like Rosy with the DB and PO ?17:29
henningen3rd: I don't know Rosy but we do have "external suggestions" in Rosetta.17:29
henningeThat is one of the key features of it, actually.17:30
henningeIf a string has already been translated in a different package or project, it's will be suggested to the translator.17:30
henningenext question17:31
cjohnston < bullgard45> QUESTION: How do you take care of that the same English phrase needs to be translated differently depending on the program at hand?17:31
adiroibanJust add a new translation17:31
henningebullgard45: that's why they are just presented as "suggestions". It still takes a reviewer to acutally accept it as a translation to use.17:31
adiroibanbullgard45: a program will use a certain PO file17:32
adiroibanand you can have the same string in 2 PO files17:33
adiroibanand for each PO file to have different translations17:33
henningein database terms:17:33
henningeYou can have multiple TranslationMessage entry related to one POTMsgSet entry. I will come to that in a minute.17:34
henningenext question?17:34
cjohnston< n3rd> QUESTION:Would it not result in overhead when there are many revisions for the same sentence, and finally refined to perfect translation?17:34
henningen3rd: it does ;-)17:35
henningen3rd: we have a lot of entries in the POTranslation table.17:35
henningebut since we consider all those refining steps as contributions we keep them for reference's sake.17:36
henningenext questions?17:36
cjohnston< n3rd> QUESTION : So the alternative is ?17:36
henningen3rd: none I am aware of atm...17:37
henningeWe have been cleaning out translations of discontinued Ubuntu series.17:37
henningeAlso, for most cases there are not that many iterations, especially if the translations are imported from upstream.17:38
henningemore questions?17:38
cjohnston< Emilien> Question: in a project that is oenly translated, if I modify an already translated string, does it overwrite the old translation, or is it stored as a new entry?17:38
henningeEmilien: new entry.17:38
henningeA new TranslationMessage and POTranslation entry is created.17:39
henningeLet me add my explanations for the remaining four tables that all deal with this:17:39
henningePOTranslation is a simple look-up table and holds the actual translated strings17:39
henningeTranslationMessage holds all the information about a translation to a specific language, like when it was done and by whom, if it was translated in Launchpad or imported from elsewhere, if it is currently used or just a suggestion, etc. For each POTMsgSet there may be multiple entries in this table, even for the same language, because any translation ever made is stored in the database, even if only the latest is actually used.17:39
henningeThe actual translation strings are referred to by their id in POTranslation.17:39
henningeLanguage is the set of all languages known in Launchpad. This table is not specific to Launchpad Translations as it is used in other parts of Launchpad, too.17:40
henningePOFile represents the set of translations into a certain language for a POTemplate. If it was created by importing a PO file, it also holds some information about that file. It is not linked directly to any translation but this relationship can be derived through the Language table.17:40
cjohnston< bullgard45> QUESTION: I have seen a lot of po files in the WWW. How are they related to Launchpad?17:40
henningebullgard45: Can you be more specific about WWW, please?17:41
henningeas in: I did not understand the question ...17:42
henningeanother question ?17:42
cjohnstonThat's it for nwo17:42
henningeadiroiban, take us into the code, please.17:42
adiroibanOK. we should see the Launchpad Translation folder structure17:43
adiroibanI will not describe them in alphabetic order17:44
adiroiban== Model ==17:44
adiroiban* _interfaces_ contains the Zope interface definitions and schema for the objects used by the application. You will find interfaces for each of the database tables described earlier. For example, potemplate.py contains IPOTemplate.17:44
adiroiban* _doc_ contains function tests for code from model. Written as doctests.17:44
adiroiban* _model_ contains objects mapping to relational database using storm.17:44
adiroiban* _tests_ contains unit tests for code from model.17:44
adiroibanNext we have the folder containing the view layer17:45
adiroiban== View ==17:45
adiroiban* _browser_ contains classes dealing with presentation logic and user interaction.17:45
adiroiban* _browser/tests_ contains unit tests for code from browser.17:45
adiroiban* _emailtemplates_ contains templates for various emails issued by Rosetta.17:45
adiroiban* _help_ contains documentation and help pages integrated with Rosetta.17:46
adiroiban* _stories_ constains functional tests for code from browser. Written as doctests.17:46
adiroiban* _templates_ contains Zope Page Templates use by the objects from browser.17:46
adiroiban* _windmill_ contains tests for javascript code using Windmill Python API.17:46
adiroiban* _lib/canonical/launchpad/javascript/translations/_ contains YUI 3 javascript code.17:46
adiroibanand finaly some utilities17:46
adiroiban== Utilities ==17:46
adiroiban* _scripts_ contains various helping scripts used in cronjobs or doing other utility and integration jobs.17:46
adiroiban* _scripts/tests_ contains tests for code from scripts.17:46
adiroiban* _utilities_ contains utility classes used in model and browser code, mostly data conversion related.17:46
adiroibanHope you have noticed that almoust all code has the coresponding tests17:47
cjohnston12:43 < bullgard45> hennige: I have seen many .po files having English text of GNOME programs and their German translations. I could make good use of some of them. I do not have a particular URL at hand  just now but I can provide one in a moment.17:48
cjohnston12:43 < bullgard45> hennige: I have seen many .po files having English text of GNOME programs and their German translations. I could make good use of some of them. I do not have a particular URL at hand  just now but I can provide one in a moment.17:48
henningecjohnston: you can find those files officially on the Damned LIes website of gnome.17:49
henningeThey are all imported into Launchpad to be included in the Ubuntu language packs.17:50
henningeWe only edit them in Launchpad Translations if we need to differ from standard Gnome.17:50
henningeMore questions?17:51
cjohnston< Emilien> QUESTION: (but maybe keep it for later, as it's not relevant right now): When there is only one series, and one translation template, why couldn't we get the template page as the default17:51
cjohnston                 translations page. Example: with project BaShare, you have 3 translation pages: root https://translations.launchpad.net/bashare , series17:51
cjohnston                 https://translations.launchpad.net/bashare/trunk/+translations and template https://translations.launchpad.n17:51
cjohnston< Emilien> et/bashare/trunk/+pots/bashare . The last page is IMHO the most useful, as it contains links to untranslated, need review or changed strings. This could be the default page when there is17:51
cjohnston                 only one series and one template... What is your opinion about that (and sorry for long question)17:51
henningeEmilien: that sounds like a good idea as long as it is clear to viewer what is going on.17:52
henningeEmilien: You can describe that in a bug, discuss it with us and even go about and fix it ... ;-)17:53
henningeMore questions?17:53
cjohnston< bullgard45> QUESTION: hennige: An example for a po file that I just have found in the WWW, is http://www.mail-archive.com/xfce4-commits@xfce.org/msg08020.html17:53
henningeThat is actually a diff of PO file ... ;)17:54
henningeThis should be imported into Launchpad if they'd want to use it for translations. https://edge.launchpad.net/xfce17:55
henningeBut AFAIR they decided against using Launchpad for translations.17:55
henningeAnother question?17:56
cjohnston < Emilien> Question: I like how all the parts have their own tests. Are they automatically executed in some way?17:56
henningeAh, the great Launchpad test suite, Emilien!17:56
henningeRun "make check" to run the whole test suite. Takes a few hours.17:57
henningeEach branch that is submitted to the LP tree needs to have had this run.17:57
henningeThen we have the buildbot that continously runs the test suite on the trunk with every new commit.17:58
henningeOne more question?17:58
cjohnston< bullgard45> QUESTION: I understood that you considered the translations provided by Launchpad as proposals or a help for a human translator. This somewhat contradicts the attitude of a German17:58
cjohnston                    translator team to admit or restrict would-be human helpers. (But I cannot provide the document that proves such a censorship.) I do understand that a high standard of qualtiy17:58
cjohnston                    translations must be maintained.17:58
henningebullgard45: I think what you are referring to is this:17:59
henningeAnybody can submit suggestions in Launchpad but only those on the translation team are reviewers and can actually accept them to be used.18:00
henningeThis requeires some deal of organization to keep the translations consistent.18:00
henningeBut actually, bullgard45, I don't see a question in there ... ;-)18:01
henningeBut, we have run out of time.18:01
cjohnstonThanks henninge and adiroiban !18:01
henningeThank you all for coming. Come and talk to us on #launchpad any time. Thanks.18:01
cjohnstonNext up is smoser18:02
smoserHello all18:02
smoserOk, lets get started.18:02
smoserThis is my first time doing one of these, so I ask for forgiveness in advance18:02
smoserI'm Scott Moser (https://launchpad.net/~smoser).18:02
smoserFor the past few months I've been working on maintaining and improving Ubuntu's presense on Amazon's Cloud Offering, known as "EC2"18:02
smoserIn this session, I hope to explain :18:02
smoser * A little bit about what EC2 is (at least from my point of view)18:03
smoser * What you need to do to use it18:03
smoser * What you can do with it18:03
smoser * Some warnings about issues you might find18:03
smoser * Some miscellaneous things I'd like to mention18:03
smoserand then, have some time for questions18:03
smoser * Questions18:03
smoserfirst off, I guess I'd like to kind of see a show of hands on who is interested in this session18:04
smoserif you are, simply a 'o/' or 'i am' ....18:04
smoserwell, seeing none , we'll just go on18:05
smoserIf you have questions along the way, please feel free to interrupt.  If it becomes too much, then we might have cjohnston help moderate.18:05
cjohnstonThere's quite a efw18:05
cjohnstonquite a few18:05
smoser== A little bit about what EC2 is (at least from my point of view) ==18:05
smoser"""Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers."""18:05
smoserWhat that means to me is:18:06
smoser* Virtual Servers, as many "clean instances" as you want, provisioned on your command18:06
smoser* ability to access permenant professional grade storage from inside the virtual servers (EBS or S3)18:06
smoser* access to fast network resources (much better than my cable modem)18:06
smoserYou can use the above, and do whatever you want, then throw it away.18:07
smoserYou get charged by the unit (hour/transfer/disk-size) for the resources as you are using them.18:07
smoserDepending on what your interest is, it may be tempting to simply calculate the charges for a month of m1.small (0.085 * 24 * 30 = $61.20), compare that to a virtual private server at http://www.linode.com and stop looking.18:08
smoserHowever, EC2 and "Cloud" is something quite different.  The biggest difference being the ability to start up and tear down instances in very little time.18:08
smoserAny questions so far?18:08
cjohnston< acicula> Question: define little, do they have spinning instances running or something?18:09
smoseracicula, 'little' is defined in terms of seconds , i'd say on average less than 10018:09
smoseri dont have hard numbers, and those vary based on time of day and load of ec2, and also (I think) popularity of the instance being requested18:10
smoser(the more popular an instance, the faster you get a copy)18:10
smosermoving on18:11
smoser=== What you need to do to use it ===18:11
smoserAssuming you weren't completely turned off from my explanation above lets move on to how you can use this.18:11
smoserI'm really only going to cover command line usage via the ec2-api-tools, but there are several other ways to manage ec218:12
smoserFirst thing you need is a credit card, and an AWS account with EC2 access.18:12
smoserThe EC2Starters Guide (https://help.ubuntu.com/community/EC2StartersGuide) has a good description on how to get signed up.  Make sure to generate, and download your X509 certificate, and key and keep them safe.18:12
smoserNext, you'll need the ec2 api tools. The latest version is available in Lucid and backports are available for karmic and hardy at https://launchpad.net/~ubuntu-on-ec2/+archive/ec2-tools . Install those tools by enabling multiverse and then:18:13
smoser$ apt-get install ec2-api-tools ec2-ami-tools18:13
smoser(note, for just "using" the cloud, the api-tools are probably enough. if you get interested in creating content for it, you will need to get the ec2-ami-tools)18:14
smoserCommands I show will assume that you have EC2_PRIVATE_KEY and EC2_CERT in your environment.18:14
smoserthe EC2StartersGuide mentions how to set that up also18:14
smoserAccess to instances "in the cloud" is via network only, and the primary method being ssh (at least here)18:15
smoserIn order to access instances via ssh, you need to create a keypair.18:15
smoserThis is a normal ssh private/public key, but amazon makes public portion available to the instance when you run it.18:15
smoserCreate that keypair with:18:16
smoser$ ec2-add-keypair ec2-keypair.us-east-1 > ec2-keypair.us-east-1.pem18:16
smoser$ SSH_KEY=$PWD/ec2-keypair.us-east-1.pem18:16
smoserAt this point, you're ready to go.  You can launch a new instance of the latest release of karmic on i386 with:18:17
smoser$ ec2-run-instances --instance-type m1.small --key ec2-keypair.us-east-1 ami-1515f67c18:17
cjohnston< bullgard45> QUESTION: What is an AWS account?18:17
smoserbullgard45, sorry, AWS Account is Amazon Web Services (http://aws.amazon.com/)18:18
smoserexpanding on m1.small18:18
cjohnston< kamalmostafa> QUESTION: Please expound on "m1.small".18:19
smoserthe above command says "give me a machine of type m1.small" and make the keypair that i just created able to access it.18:19
smoserm1.small is described at http://aws.amazon.com/ec2/instance-types/18:19
smoserand pricing information available at http://aws.amazon.com/ec2/#pricing18:20
smoserm1.small is the cheapest, and is i386 virtual hardware18:20
smoserYou will see output similar to the example at https://wiki.ubuntu.com/UbuntuDeveloperWeek/Sessions/GettingStartedWithEC2 .18:21
smoserThe magic 'ami-xxxxxx' number above is the image-id. You can see a list of available images via:18:21
smoser$ ec2-describe-images --all > images.list18:21
smoserI redirected output to a file because the list is quite long.  Another interface for looking at available machine images is http://thecloudmarket.com/18:22
smoserOfficial Ubuntu images are named such that you can easily identify them.  For more information on that see: https://wiki.ubuntu.com/UEC/Images/NamingConvention18:23
smoserYou'll note that after creation, you got an 'instance id' for the new instance (i-xxxxxx)18:23
smoserand that it was in 'pending' state.18:24
smoserAt some point, your instance will move from 'pending' state to 'running' and will have a hostname associated with it.18:24
smoseryou can watch the state with18:24
smoser$ ec2-describe-instances18:24
smoserOnce you see 'running' and a hostname, then you can ssh to the instance18:24
smoser$ ssh -i ${SSH_KEY} ubuntu@ec2-174-129-61-44.compute-1.amazonaws.com18:24
smoserUbuntu images accept only the 'ubuntu' user out of the box18:25
smoserand that user has passwordless sudo configured.18:25
smoserHoorah! now you've got root access to your very own machine in the cloud, and it cost you $0.085 USD18:25
smosernow, you can shut that instance down with 'shutdown' or18:26
smoser$ ec2-terminate-instances i-xxxxxx18:26
smoserwe'll pause for questions on the above18:26
smoser(i realize theres a lot for "getting started")18:26
smoserok. going on then.18:27
smoser=== What you can do with it ===18:27
smoserSo, now that you've created this new machine and have access to it, what can you do?18:27
smoser * Test : We make night nightly builds of the development release, and semi-nightly builds of previous releases available.18:27
smoserThis makes it extremely easy to verify a change in a development release without any risk to your system18:28
smoser * develop18:28
smoserIf you don't have access to hardware supporting kvm, or just don't want to bother with keeping multiple images around of old releases or development, you can easily start up an instance, and start developing on it.18:28
smoserWhen you're done, you just throw it away.  A new one is only seconds away.18:28
smoserAs an ubuntu developer, those two items above are the most interesting to me.  My laptop is not normally in a situation where it has a clean environment available for test or build.18:29
smoser * Make your own images / devpay18:29
smoserYou can make your own images and register them, and then boot your personal image.18:29
smoserYou can "Make Money!" using DevPay (http://aws.amazon.com/devpay/).  If you add value to an image, enough that people will pay to use it, its easy to charge them for using their image.18:29
smoserI've not experimented at all with devpay, but it is a really easy way that you can make software appliances available to others and get paid!18:30
smoserI'll pause again for questions...18:30
smoser=== Some warnings about issues you might find ===18:31
smoserThe thing that most sticks out as different from other environments is that the root filesystem is *gone* once you shut a system down.18:31
smoserAmazon makes basically no promises for data stored on the "ephemeral storage" of an instance.18:31
smoserhardware failure that causes an instance to die would cause data there to be lost18:32
smoserIf you want to keep data that you create on an instance, you have a few options:18:32
smoser * Use S3 storage (from amazon)18:32
smoser * Use EBS (and connect an ebs volume to your instance with ec2-attach-volume)18:32
smoser * Use a (new) EBS Root instance.18:32
smoserIn this new type of instance, your / is on a EBS volume, and you can stop your instance, and access that volume from another instance.18:32
smoserEBS Root Instances make the EC2 cloud much more similar to a "pay as you go Virtual Private Server"18:33
smoserWe are planning on creating EBS volume IDs for Lucid, and Eric Hammond has made EBS boot AMIs available for Karmic and Hardy (http://alestic.com/2010/01/public-ebs-boot-amis-for-ubuntu-on-amazon-ec2)18:33
smoserThe "best practice" for using ec2 instances is to make all setup of your environment automated18:34
smoserso that you can start up new instances on failure or demand.18:34
smoserpause again. questions ?18:34
smoser=== Some Miscellaneous Things I'd like to mention ===18:35
smoser * User Data18:35
smoserBooting an exact copy of an instance over and over again is hardly any fun, and arguably not much use18:35
smoserIn almost all cases, you will want to make some modifications to the instance immediately after boot.18:35
smoserYou can do this by waiting util the machine is running, and ssh'ing in and running a set of commands.18:36
smoserBut that may be less than idea.18:36
smoserThis is where "User Data" comes in.  User data is provided to the instance via the '--user-data' or '--user-data-file' arguments to ec2-run-instances.18:36
smoserWhen the instance boots, it reads the user-data an can change its behavior based on that content.18:37
smoserWhen an ubuntu instance starts, if user-data begins with '#!' then it will be executed like an rc.local script.18:37
smoserThis means that you can put together a shell script to turn the "base AMI" into your personal development box and pass that script as user-data to the instance on startup.  Then, when you ssh in, it will be all set up.18:38
smoserThis is obviously quite useful for all sorts of purposes18:38
smoserWe're adding features to how user-data is handled for lucid (https://wiki.ubuntu.com/ServerLucidCloudBoothooks and https://wiki.ubuntu.com/ServerLucidCloudConfig for more information).18:38
smoserThe big new feature will be the ability to configure the image on first boot via "config"  syntax rather than a shell script.18:38
smoser * Spot Instances18:39
smoserIf you're like me, you're cheap.  Spot instances allow you to run an instance in basically the same way as "normal instances" but at a reduced price.18:39
smoserThe catch is that if there is high demand, your instance may be terminated.18:39
smoserI've began using spot instances for most of my tests.18:39
smoserWhen you make a request for a "spot instance" you state the maximum you will pay for that instance.  The rate you are charged varies based on demand for those resources.18:40
smoserif the rate goes above your maximum, your instance is terminated.18:40
smoser * Desktop In the Cloud18:40
smoserIn the lucid cycle, the Desktop team wanted to take advantage of the easily created test/reproduce/verify environment18:41
smoserso we started creating "desktop in the cloud" builds.18:41
smoserThose are created daily and registered to ec218:41
smoserhttp://uec-images.ubuntu.com/desktop/lucid/ for relevant AMIs.18:42
smoserThese images have a full gnome desktop installed and can be accessed via an NX client.  If you've never used NX before, it is surprising how well it performs.  Literally, you may forget that you are using a remote desktop.18:42
smoserhttp://www.stgraber.org/2009/12/12/ubuntu-desktop-cloud-now-working-lucid has information on how to use these.18:43
smoserQUESTION: smoser, when you use spot instances for your testing, what rate do you typically offer to pay, and how often do your instances get terminated?18:43
cjohnston< kamalmostafa> QUESTION: smoser, when you use spot instances for your testing, what rate do you typically offer to pay, and how often do your instances get terminated?18:43
smoserI've never actually had one terminated.18:44
smoserhttp://cloudexchange.org/ has graphs of spot prices historically18:44
smoserand you can get that same data with : ec2-describe-spot-price-history18:44
smoserwhat I do is take the high price available and raise it by 10%18:45
smoseryou'll notice spikes in the graphs at cloudexchange18:45
smoserthis is where (I think) amazon had higher demand for non-spot instances and explicitly kicked off most spot instances18:46
smoserOn average, spot instances run ~ 40% of the regular price.18:46
smoserThis is all I had prepared, so the floor is open to questions18:47
smoseroh. One other thing I should mention18:48
smoseri can't believe i didn't plug UEC18:48
smoserUEC (Ubuntu Enterprise Cloud) is based on Eucalyptus and allows you to easily run a ec2 compatible cloud inside your own data center.18:48
smoserThis has all sorts of neat uses.18:49
cjohnston< zul> QUESTION: how are the images built?18:49
smoserzul, thanks for asking :)18:49
smoserWe build our images using vmbuilder18:49
smoserThe complete setup for building identical images is available starting at : https://code.launchpad.net/~ubuntu-on-ec2/vmbuilder/automated-ec2-builds18:50
smoserand code we use to publish the builds is at https://code.edge.launchpad.net/~ubuntu-on-ec2/ubuntu-on-ec2/ec2-publishing-scripts18:50
cjohnston< dhillon-v10> smoser: how is the upcomming HTML5 going to affect the server cloud18:51
smoserhm.. I dont know.18:51
smoserDo you think it will increase demand for server resources? or reduce demand?18:51
smoserI guess I need more context on the question18:52
cjohnston< dhillon-v10> smoser: probably decrease the demand because now users can do more stuff offline18:52
smoserMaybe, but by the same note, the more you depend on "the cloud" the more important reliable resources are18:53
smoserWell, I guess to wrap up...18:54
cjohnston< dhillon-v10> smoser: possibly :)18:54
cjohnston< kamalmostafa> QUESTION: Can EC2 machine images be used by UEC installations?  (either with our without incurring charges at Amazon)?18:54
smoserkamalmostafa, yes18:54
smoserThe only real difference is that UEC uses KVM and EC2 is xen based.18:55
smoserso the end result is that you'll have a different kernel, but ubuntu creates "ubuntu sauce" kernels for both varieties18:55
smoseri see UEC as very useful for developing machine images that can then be deployed publically on ec218:56
smoser(and for other purposes too)18:56
cjohnston< kamalmostafa> QUESTION: Does one need an Amazon AWS account at all, in order to use those images in a UEC installation?18:56
smoserkamalmostafa, no18:56
smoserUEC is completely stand alone.18:56
smoserThe requirements for it are basically just having hardware available.18:56
smoseryou can create a "cloud" with 2 systems18:56
smoserand the ubuntu-server ISOs have an easy option for doing so.18:57
smoserI've probably represented UEC poorly here... maybe next developer week we need a UEC class18:57
smoserwrapping up...18:58
cjohnston< lbrinkma> QUESTION: Whats the api command for creating a spot instance?18:59
smoserIf you're a ubuntu developer, I strongly urge you to take a look at using EC2 for easily running releases of ubuntu that you have no worry about destroying.18:59
smoserec2-request-spot-instances is the api command18:59
smoserit basically looks like 'ec2-run-instances' with some extra args.18:59
smoserI've done sone work to make a wrapper 'ec2-cheap-run-instances'.  It needs some work, but then is a dropin replacement for ec2-run-instances19:00
smoserIf you're not a ubuntu developer but want to do "real stuff" with EC2, there are loads of people running businesses and servers on EC2's cloud.  I urge you to look into it further, and,of course, I urge you to start with the Official Ubuntu Images19:01
smoserIf you have questions or need assistance, please feel free to ping me  #ubuntu-server or elsewhere.19:02
cjohnstonThanks smoser !19:02
cjohnstonkirkland, your next19:03
kirklandcjohnston: hi, thanks.19:03
kirklandalright, so I'm here to talk about KVM19:03
kirklandwhich is the recommended virtualization hypervisor for Ubuntu19:03
kirklandit's similar, at least in principle to other hypervisors like VMWare, Xen, or VirtualBox (among many others)19:04
kirklandthis talk is structured as follows ...19:04
kirkland a) a brief description/explanation about Virtualization, and how KVM works19:04
kirkland b) a few simple examples for launching virtual machines with KVM (that you can try at home now)19:05
kirkland c) and finally, some "KVM hacks", to do some fun, or more interesting things with your VMs19:05
kirklandso for starters, check to see if your CPU supports KVM19:06
kirklandegrep "flags.*:.*(svm|vmx)" /proc/cpuinfo19:06
kirklandyou need to see if your CPU supports either the svm or the vmx flag19:06
kirklandthese are the AMD and Intel designations for hardware supported virtualization19:06
kirklandquick poll in #ubuntu-classroom-chat, hands up o/ if you have cpu support for kvm19:07
kirklandif you don't have hw support for KVM, you'll probably need to use something else (like Xen or VirtualBox) to do virtualization19:07
kirklandunfortunately, I'm not going to cover those in this talk19:08
kirklandokay so KVM ...19:08
kirklandkvm is actually supported through a driver in the linux kernel19:08
kirklandso if you have cpu support, you can "sudo apt-get install qemu-kvm"19:08
kirklandonce that installs, you should have a device, /dev/kvm19:09
kirklandthis is an interface that the kernel provides to the userspace kvm program19:09
kirklandbasically, this allows virtual machines to runs some instructions *directly* on your real cpu19:10
kirklandrather than running those instructions on an emulated CPU19:10
kirklandwhich is what qemu alone (ie, without kvm) does19:10
kirklandneedless to say, emulation is slow19:10
kirklandrunning on real hardware is much faster!19:10
kirklandfor this reason, KVM is really smoking fast virtualization19:10
kirklandhowever, we still need parts of QEMU19:11
kirklandQEMU = "quick emulator"19:11
kirklandit provides what we call "the driver model" for KVM19:11
kirklandit emulates the rest of the things that aren't accelerated (yet)19:11
kirklandlike the sound card, and the video card19:11
kirklandso there are a lot of ways to launch virtual machines with kvm19:12
kirklandyou can launch them through the command line19:12
kirklandwhich is what I generally do, as a developer19:12
kirklandor through one of the graphical front ends19:12
kirklandvirt-manager is probably the most popular front end19:13
kirklandi encourage you to try virt-manager out, if gui's are your thing19:13
kirklandit's pretty straight-forward, wizard/menu driven19:13
kirklandit prompts you for various options about your vm creation19:13
kirklandand then, ultimately, just runs a really long kvm line :-)19:14
kirklandit does all of this using the libvirt library19:14
kirklandlibvirt provides a common library interface to multiple different virtualization backends (like kvm, xen, and others)19:14
kirklandto get a quick example up and running, let's use "testdrive"19:14
kirklandanyone here already running Lucid on the machine they're going to use to try out these examples?  (o/ hands up in -chat)19:15
kirklandif so, just "sudo apt-get install testdrive"19:15
kirklandotherwise, you'll need to install testdrive from the PPA linked to from http://launchpad.net/testdrive19:16
kirklandtestdrive is just a wrapper that rsync's or zsync's an Ubuntu ISO and launches it in a KVM with a good set of options19:16
kirklandit's a nice way of "testdriving" the daily Ubuntu desktop or server ISOs in a virtual machine19:17
kirklandhopefully you have an Ubuntu *.iso file lying around somewhere19:18
kirklandotherwise, you're going to need to download one to do any of the following examples19:18
kirklandfortunately, I do ...19:18
kirklandand testdrive can either run against a http/ftp/rsync able ISO19:18
kirklandor against a local file19:18
kirkland$ testdrive -u ./lucid-desktop-amd64.iso19:19
kirklandso that's what I'm running right now19:19
kirklandand so the desktop live cd installer then pops up in a new window19:19
kirklandlet's look at what KVM was launched with19:20
kirkland$ ps -ef | grep kvm19:20
kirklandkirkland 11395 11364 99 13:19 pts/12   00:00:35 kvm -m 512 -cdrom /local/virt/iso/lucid-desktop-amd64.iso -drive file=/home/kirkland/.cache/testdrive/img/testdrive-disk-nvqebh.img,if=virtio,index=0,boot=on -usb -usbdevice tablet -net nic,model=virtio -net user -soundhw es137019:20
kirklandthe options are mostly straightforward19:20
kirklandbut testdrive launched with a decent set of options without you having to go figure all of these out19:20
kirklandbut since you're here to learn about KVM, let's look at them  :-)19:21
kirkland-m 512 says give this virtual machine 512MB of memory19:21
kirklandwhich is enough to use the LiveCD19:21
kirklandmy VM is now up and running the liveCD19:21
kirklandi can launch a terminal in the VM and run "free"19:21
kirklandand see that yes, this VM has ~512MB of memory19:21
kirkland-cdrom tells KVM where to find the ISO to boot19:22
kirkland-drive tells KVM where to find the hard disk19:22
kirklandnow there's a bit more to this19:22
kirklandin particular, take note of if=virtio19:22
kirklandthis VM is using "virtio" for the hard disk driver19:22
kirklandthis is *much* faster for KVM to use, than say, scsi19:23
kirklandvirtio is a "paravirtual" disk driver19:23
kirklandkvm is able to write to the backing disk image more directly, through a special, virtio feature in the host's kernel19:23
kirklandyou can also see where the .img file is stored (in ~/.cache/testdrive/*img in this case19:23
kirklandit's index=0 which puts this disk at /dev/vda19:24
kirkland(note that the disk would be /dev/sda if the if=scsi19:24
kirklandand boot=on makes kvm able to boot from this disk19:24
kirkland-usb -usbdevice tablet is just a neat little hack that allows you to move your mouse cursor in and out of the KVM sdl window more smoothly19:25
kirklandtotally optional, but I really like it19:25
kirkland-net nic,model=virtio is the network adapter19:25
kirklandagain, we're using virtio for networking19:25
kirklandwhich, like virtio disk, is a paravirtual networking driver, allowing the vm to talk through the host's network adapter *much* faster19:26
kirklandlike 10x faster19:26
kirkland-net user is the type of networking, there are a lot of different options here, but this is the simplest for quick outgoing network access19:26
kirklandand -soundhw is the emulated sound device19:26
kirklandokay ... was anyone able to get a desktop LiveCD VM launched (either through testdrive or on your own)?  hands up in -chat o/19:27
kirklandah, okay, I see a question about KVM in the BIOS19:27
kirklandlet me address that ....19:27
kirklandfirst, run "kvm-ok" on your host19:28
kirklandthat should help detect *most* issues why KVM might not work19:28
kirklandfirst, it does that grep I asked you to do earlier19:28
kirklandnext, it does a few sanity checks, including grepping for a really annoying message in BIOS19:28
kirklandsome desktop/laptop manufacturers ship machines that *have* CPUs that support virtualization (VT)19:29
kirkland*but* they disable it in BIOS19:29
kirklandthe widespread rumour is that M$ pressures them to do so19:29
kirklandin any case, we all suffer with this19:29
kirklandso if you enter your bios, look in CPU options, and you should see something about "enabling VT, Virtualization Technology"19:29
kirklandturn that on, physically power off, then back on19:30
kirklandand then re-run kvm-ok, and you should be good to go19:30
kirklandcool, looks like we have a few people with running VMs19:30
kirklandnow, go ahead and perform an installation (if you like)19:30
kirklandand your desktop image should be installed into that single disk image19:30
kirklandit'll take a little while to complete19:31
kirklandokay, mine's going in the background19:32
kirklandthe fun thing is that you can do *anything* in this VM without affecting your stable desktop19:32
kirklandfor that reason, KVM is wonderful for developers and testers19:32
kirklandi did all of my development of Ubuntu's Encrypted Home Directory feature, for instance, in hundreds of VMs that I created and destroyed19:33
kirklandhave you ever wanted to cd / && sudo rm -rf / ?19:33
kirklandbe my guest, in your VM :-)19:33
kirklandanyway, you can do some really constructive (or destructive) things in VMs19:33
kirklandlet's pause for a few questions, now, while our installations are going, and before we get into some advanced features19:34
kirklandplease post your questions in #ubuntu-classroom-chat, prepended with: QUESTION19:34
kirkland<rmunn|lernid> QUESTION: If I have an existing VM created in VirtualBox and/or VMWare, is it possible to convert it to run under qemu-kvm instead?19:34
kirklandrmunn|lernid: possibly ...  or at least you can get very, very close19:35
kirklandrmunn|lernid: mainly, you need to track down the disk image, where it installed into19:35
kirklandrmunn|lernid: assuming it's a standard disk format, you should be able to launch19:35
kirklandrmunn|lernid: you might have to recreate some of the meta data, though19:36
kirklandrmunn|lernid: like how much memory the VM gets19:36
kirklandrmunn|lernid: but if you find the disk image itself, make a backup copy of the disk image (while the VM is *not* running), cp it to /tmp/foo.img, for instance19:36
kirklandrmunn|lernid: and then just run "kvm -m 512 -hda foo.img"19:36
kirklandrmunn|lernid: if that works (or mostly works), you'll then just need to play with getting the rest of the kvm options you want in there (like adding networking, sound, etc)19:37
kirklandrmunn|lernid: as for something more automated, it might well exist ... though I don't know of it19:37
kirklandrmunn|lernid: i think that would be a cool idea19:37
kirklandrmunn|lernid: file a brainstorm request, and/or a wishlist bug19:37
kirkland<rmunn|lernid> QUESTION: How do you save a "snapshot" of your VM's state under qemu-kvm, then revert to that snapshot later, discarding any changes since the snapshot?19:38
kirklandrmunn|lernid: *great* question ... i'll try to get to that in the advanced examples section19:38
kirklandrmunn|lernid: in case I don't get all the way through to it (as I didn't prepare that particular example), see: http://manpages.ubuntu.com/manpages/lucid/en/man1/kvm.1.html19:39
kirklandrmunn|lernid: search in that manpage for snapshot=19:39
kirklandrmunn|lernid: and -snapshot19:39
kirkland<kjele> QUESTION: What benefit does kvm have over virtualbox?19:40
kirklandkjele: for one thing, KVM is really, really, really fast19:40
kirklandkjele: though it requires hardware support to be really, really, really fast19:40
kirklandkjele: KVM is mostly maintained by the Ubuntu Server team19:40
kirklandkjele: and our target hardware (mostly server class hardware) has such support pretty ubiquitously19:41
kirklandkjele: personally, I think VirtualBox is the best alternative for users who *don't* have hardware support19:41
kirklandkjele: and VirtualBox has a pretty GUI19:41
kirklandkjele: ideally, someone in the Ubuntu Desktop team would perhaps support VirtualBox19:42
kirklandkjele: but as of now, it's a project in Ubuntu Universe that mostly works pretty well19:42
kirkland<yltsrc> QUESTION: is kvm faster than virtualbox and vmware?19:42
kirklandyltsrc: in my experience, and on hardware that has VT, yes, and yes.19:42
kirklandyltsrc: in particular, if KVM is using virtio networking and disk, yes, and yes ;-)19:43
kirkland<kamalmostafa> QUESTION: Does testdrive create a blank .img of a particular size before booting the specified iso?  Or does the .img file somehow "grow" as needed?19:43
kirklandkamalmostafa: good question19:43
kirklandkamalmostafa: yeah, it totally does, and I'm glad you reminded me to mention the command19:43
kirklandkvm-img create -f qcow2 $DISK_FILE $DISK_SIZE19:43
kirklandthat's the command testdrive uses19:43
kirklandif you're creating/launching custom VMs outside of testdrive, you'll need to use "kvm-img create" to create a backing disk image19:44
kirklandthere are several supported -f formats19:44
kirklandusually, I use either "raw" or "qcow2"19:44
kirklandI strongly recommend "qcow2" whenever possible19:45
kirklandqcow = quick copy on write format 219:45
kirklandwhich allows the backing disk image to be *much* smaller than the total allocation19:45
kirklandtestdrive randomly generates the filename19:45
kirklandand uses 6G for the backing disk size, by default19:45
kirkland(both of which can be overriden by the user (see the testdrive manpages)19:46
kirklandokay, my install has completed19:46
kirklandhas anyone else's installation completed?  (hands up, or % done in -chat)19:46
kirkland<lernid_luislopez> QUESTION: Is it possible to have live migration in KVM?19:46
kirklandlernid_luislopez: yes!  we'll try that out shortly, actually :-)19:47
kirkland<yltsrc> QUESTION: is kvm ready for production?19:47
kirklandyltsrc: another good question ...19:47
kirklandyltsrc: i would say "if you're on a modern version, yes" :-)19:47
kirklandyltsrc: let me qualify that ...19:47
kirklandyltsrc: Ubuntu 8.04 LTS (Hardy) was the first of the enterprise distros to ship with KVM as the hypervisor19:48
kirklandyltsrc: however, the kvm version back then (and more importantly the kernel support) was very young19:48
kirklandyltsrc: for this reason, I backported kvm-84 to hardy-backports19:48
kirklandyltsrc: I'd say that version (which is Jaunty's KVM) is pretty good19:48
kirklandyltsrc: however, Karmic's (and Lucid's) is far more stable19:48
kirklandyltsrc: and KVM is ultimately what UEC (Ubuntu Enterprise Cloud and Eucalyptus) are using for cloud VMs19:49
kirklandyltsrc: which is production ready ;-)19:49
kirkland<statik-lernid> QUESTION: I used testdrive-select-iso last night and ran through a lucid desktop installation, but then i turned it off this morning. is there a way for me to get back to that machine without having to do a whole new install?19:49
kirklandstatik-lernid: if you want to launch the backing image, yeah, sure ... you just need to find the disk image19:50
kirklandstatik-lernid: cd ~/.cache/testdrive19:50
kirklandstatik-lernid: cd ~/.cache/testdrive/img19:50
kirklandstatik-lernid: ls -thalF *.img19:50
kirklandstatik-lernid: look at the timestamps, and the file sizes, and you should be able to find the one you want19:50
kirklandstatik-lernid: and then just launch KVM19:51
kirklandokay, let me tear through the rest of this19:51
kirklandI have a few fun hacks I'm going to just paste and tell you about, as I'm running low on time19:51
kirklandkvm -m 1024 -cdrom http://cdimage.ubuntu.com/daily/current/lucid-alternate-amd64.iso19:51
kirklandthis one is fun ...19:51
kirklandyou can actually stream the ISO over http, without downloading it :-)19:51
kirklandi have a pretty good ISO mirror here local on my gigabit network19:51
kirklandwhich is obviously faster than going over the internet19:52
kirklandso I can launch ISOs without having them locally on my small SSD hard drive in my laptop19:52
kirklandfor *really* fast VMs, put your ISO and/or hard disk image in a tmpfs in memory19:52
kirklandi use tmpfs for my /tmp19:52
kirkland(you have to have a lot of RAM, really, to do this well)19:53
kirkland    /etc/fstab:tmpfs /tmp tmpfs rw19:53
kirklandthat's the line you add to /etc/fstab19:53
kirklandkvm -m 2048 -cdrom /tmp/lucid-desktop-amd64.iso19:53
kirklandthat launches an ISO for me, with a *lot* of memory19:53
kirklandwhere the backing ISO is itself in my RAM :-)19:53
kirklandneedless to say, it's pretty quick :-)19:53
kirklandkvm-img can create disks of *any* size19:54
kirklandand qcow2 will make them sparse19:54
kirklandso you can create, let's say, a 2PB (petabyte) disk19:54
kirklandkvm-img create -f qcow2 /tmp/petabyte.img 1024T19:54
kirklandlaunch a KVM with that as the backing disk and do a "sudo fdisk -l" in the VM19:54
kirklandit's fun to see a 1PB backing disk19:54
kirklandlong before they're on the market :-D19:55
kirklandyou can dynamically add a disk to a running VM19:55
kirklandin the vm, "sudo modprobe acpiphp"19:55
kirkland(note that we're enabling this by default in Lucid, so that you don't have to modprobe it on the server)19:55
kirklandnow, in your running KVM, drop to the QEMU console, where there are all sorts of fun, interesting commands19:56
kirklandthat'll put you at a QEMU prompt19:56
kirklandpci_add auto storage file=/tmp/petabyte.img,if=virtio19:56
kirklandthen go back to your VM19:56
kirklandsudo fdisk -l19:56
kirklandthat should show you your very big disk :-)19:57
kirklandfinally, let's do a live migration ...19:57
kirklandnow we'll do the live migration from one VM to another VM, running on the same system (your localhost)19:57
kirklandbut if you have two machines running the same KVM version, you should be able to do this across your network too19:58
kirklandso first, launch a vm19:58
kirklandkvm -m 512 -hda /local/virt/img/lucid-desktop.img19:58
kirklandor use your running one19:58
kirklandnext, launch a second kvm process WITH THE EXACT SAME PARAMETERS19:59
kirklandplus the -incoming option19:59
kirklandkvm -m 512 -hda /local/virt/img/lucid-desktop.img -incoming tcp:
kirklandand the IP address and Port19:59
kirklandi'm using local, so I have
kirklandand I arbitrarily chose 102519:59
kirklandIn the source, drop to the QEMU console19:59
kirklandand run:19:59
kirkland    migrate tcp:localhost:102519:59
kirklandand in a few seconds, you should see your vm move from one process to the other20:00
kirklandto do this across separate hosts, you need to have persistent storage across both (like NFS)20:00
kirklandand it requires a slightly more complicated networking setup20:00
kirklandbut it's quite doable20:00
kirklandokay, I think my time is up20:00
kirklandhope this was useful20:00
kirklandi'll be around in #ubuntu-server if you have more questions20:01
DktrKranzHello everybody, and thank you to be here at Python Application packaging hosted at Ubuntu Developer Week!20:05
DktrKranzMy name is Luca Falavigna, I'm a MOTU and Debian Developer, and I'll be your SABPDFT (Self-Appointed Benevolent Python Dictator For Today) for the next minutes :)20:05
DktrKranzFor those who weren't here last week, I already had a session about basic Python application packaging. You can find logs here: http://irclogs.ubuntu.com/2010/01/21/%23ubuntu-classroom.html20:06
DktrKranzIt basically covered how to handle control files in debian/, and how to easily find dependencies for your Python applications. Even if you are good in Python packaging, there could be some interesting tips to improve your kung-fu :)20:06
DktrKranzToday we will look at some advanced tricks to improve overall Python packages quality and stability.20:07
DktrKranzBut let's start with a really short introduction.20:07
DktrKranzI refer to a "Python application" as a piece of software composed by one or more Python scripts and eventually one of more modules (or "packages", distutils calls them that way).20:08
DktrKranzOther kinds of software are "Python modules", which contain modules potentially useful for other programs installed in the global namespace, and "Python extensions", which usually are C source files, compiled and liked for a given Python interpreter to extend its features.20:08
DktrKranzBut how does a Python application look like?20:09
DktrKranzI've created a really dumb one for the occasion. Open your favourite terminal and launch dget -u http://people.debian.org/~dktrkranz/pythontest/pythontest_0.1-1/pythontest_0.1-1.dsc20:09
DktrKranzOnce you've downloaded it, look at pythontest-0.1 directory, you will find pythontest script, PythonTest module (or "package") and setup.py, which is Python distutils' "makefile".20:10
DktrKranzIt's probably the smallest Python application you'll ever seen, it just prints a message, and then exits :)20:11
DktrKranzInvest a couple of minutes to examine the few files available, then we will move to packaging.20:11
DktrKranz< Omar871> QUESTION: Will "setup.py install" work?20:12
DktrKranzOmar871: it should by launching "python setup.py install"20:12
DktrKranzI didn't test that way, but it should work as expected ;)20:13
DktrKranz< Omar871> QUESTION: How do I use the package after installing it?20:15
DktrKranzit installs a script under /usr/bin, named pythontest. you should be able to launch it, but we will see it in the next minutes20:16
DktrKranz< statik-lernid> QUESTION: when would we choose dh --with-python-central instead of the default?20:16
DktrKranzstatik-lernid: it's a matter of tastes. I personally prefer python-support, I read the code and I understand a bit its logic.20:17
DktrKranz< lucio> QUESTION: what is pycentrak and pysupport and why do i care?20:18
DktrKranzlucio: these are helpers which manage to expand ${python:Depends} to the right values, and byte-compile Python files during package install20:19
DktrKranzthey do more, but that's beyond the scope of the lesson.20:20
DktrKranzlucio: if you are interested, a more detailed description of packaging is held in the logs of the other session, see above :)20:20
DktrKranzok, let's move to the packaging itself now20:21
DktrKranzCompile the package (dpkg-buildpackage -uc -us) and install the resulting pythontest_0.1-1_all.deb20:21
DktrKranzTo test it, open another console and move to a completely different location than the build directory. Keep this in mind: never test a Python software installed system-wide by launching it in the build directory. This will save you from errors, and let you discover serious problems in your packaging!20:22
DktrKranzNow launch "pythontest". You should now see a message on your console saying package works correctly.20:23
DktrKranz< Omar871> QUESTION: Is debhelper needed for this?20:25
DktrKranzOmar871: for this examples, yes. debhelper is explicitly listed in Depends field in debian/control20:25
DktrKranz< statik-lernid> QUESTION: when I dpkg -L pythontest, I see files installed to /usr/share/pyshared/. when I python -c "import PythonTest;print20:27
DktrKranz                       PythonTest.__file__" I see files located at /usr/lib/pymodules/python2.6/PythonTest/__init__.pyc. why the different paths?20:27
DktrKranzstatik-lernid: good question. This is how python-support worsk20:28
DktrKranzit byte-compiles files for every supported Python version at runtime, and stores files under /usr/lib/pymodules/python*/PythonTest20:28
DktrKranzwhile package only has them under pyshared, because they are common to multiple Python versions at the same time20:29
DktrKranzthat's the magic behind python-support and python-central :)20:29
DktrKranz< POX> QUESTION: shouldn't Python applications use private dirs by default (i.e. when other packages do not use their modules)? What's --install-lib20:30
DktrKranzand --install-scripts?20:30
DktrKranzPOX: yes. Stay tuned for that :)20:30
DktrKranzEven if package works, we installed PythonTest module in Python global namespace, which is completely useless for other packages in this case. Nobody will ever want to access our PythonTest module, and we should also want to avoid collisions with other modules as well.20:31
DktrKranzTo avoid global namespace pollution, we should install our PythonTest module in a "private directory", that is one not listed in the output shown by python -c "import sys; print sys.path" command.20:31
DktrKranz/usr/share/packagename is a very good choice in almost all cases, so we'll adjust our package to install modules in /usr/share/pythontest.20:32
DktrKranzI prepared a second package, you can grab it with dget -u http://people.debian.org/~dktrkranz/pythontest/pythontest_0.1-2/pythontest_0.1-2.dsc, and see changes I've made here: http://people.debian.org/~dktrkranz/pythontest/0.1-1_0.1-2.patch20:33
DktrKranzBasically, the only change I did is passing --install-lib=/usr/share/pythontest to distutils to instruct it to install PythonTest in our private directory (see the three rules files to see several methods to do that).20:34
DktrKranzAgain, compile, install the resulting pythontest_0.1-2_all.deb, and test it by launching pythontest in the other console.20:34
DktrKranzYou should now see something unexpected...20:34
DktrKranzI got "Uh-oh! There is something wrong with the package, I can not import PythonTest module!", so we probably did something bad here :)20:35
DktrKranzBefore explaining that, let's answer to some questions.20:37
DktrKranz< mhall119|work> QUESTION: so the dh $@ in debian/rules is processing the pythontest.* files?20:37
DktrKranzmhall119|work: it's dh_pysupport (or dh_pycentral), called by dh sequencer which parsed "dh $@"20:38
DktrKranzyou can see dh_pysupport has been called in your terminal output20:39
DktrKranz< Omar871> QUESTION: What about virtual env?20:40
DktrKranzOmar871: we don't usually use virtualenv to avoid shipping multiple files in the same package20:40
DktrKranz< mhall119|work> QUESTION: so I understand, dh->dh_pysupport->pythontest.postinst.debhelper20:41
DktrKranzmhall119|work: exactly. dh calls dh_pysupport at the given time, while dh_pysupport also generates *postinst files to handle byte-compilation of Python objects.20:42
DktrKranz< johnsgruber> QUESTION: In my case for the first example source package the built deb says to put the library in /usr/share/python-support/pythontest20:43
DktrKranzjohnsgruber: do you have a python-support version << 0.90.0 ?20:43
DktrKranze.g, you use an older Ubuntu/Debian release than Intrepid/Karmic, or testing/sid?20:44
DktrKranzBack to our problem then.20:46
DktrKranzWhy did we get that? We moved PythonTest to a private directory, and this is good, but our pythontest script is no longer able to access it, and this is bad!20:46
DktrKranzThis is why PythonTest is not in global namespace anymore, but pythontest script obviously thinks it is there.20:46
DktrKranzWe have two options now: using sys.path.append() from inside of pythontest script (the one installed in /usr/bin), or moving it to the same directory where PythonTest is (/usr/share/pythontest).20:47
DktrKranzMy preferred way to achieve this goal is the latter, I don't like playing with sys.path :)20:48
DktrKranzLet's download today's third package with dget -u http://people.debian.org/~dktrkranz/pythontest/pythontest_0.1-3/pythontest_0.1-3.dsc and look at the diff at http://people.debian.org/~dktrkranz/pythontest/0.1-2_0.1-3.patch20:48
DktrKranzThis time I passed a new option to distutils: --install-script=/usr/share/pythontest, to instruct it to install our scripts in /usr/share/pythontest too. I also created a /usr/bin/pythontest symlink pointing to /usr/share/pythontest/pythontest, this way we still have pythontest in our $PATH :)20:49
DktrKranzAgain, compile, install the resulting pythontest_0.1-3_all.deb and test it by launching pythontest in the other console.20:49
DktrKranzNow you should see something familiar, and more friendly :)20:50
DktrKranzWe have PythonTest module in a private directory, and pythontest is able to import it without problem, let's celebrate! \o/20:51
DktrKranzIt's a little tricky at times, but you just need some practice, and things will be clearer.20:51
DktrKranz< mhall119|work> QUESTION: applications should always be put in a private space?20:51
DktrKranzmhall119|work: it's not always possible, sometimes modules are required by other packages, and having it in global namespace helps, but normally the private directory should be used20:52
DktrKranzin my memory, > 95% Python applications can be installed using private directory method20:53
DktrKranz< Omar871> QUESTION: Can you please explain to me what exactly is a "private space"?20:54
DktrKranzOmar871: a private directory is a directory not accessible by Python global namespace.20:55
DktrKranzOmar871: for example, if you launch python console and import a module (f.e. gtk, but feel free to pick another one), python interpreter will look in some directories looing for gtk module20:56
DktrKranzthese are the ones listed by "python -c "import sys; print sys.path" command20:56
DktrKranzother ones are "private", given that python interpreter won't look in those searching for modules20:57
DktrKranz< POX> QUESTION: what if script and module name is the same? What's --install-scripts?20:57
DktrKranzPOX: in that case, one of the quickest solutions is to manually install scripts having the same name as modules/packages (e.g. foo -> foo.py)20:58
DktrKranzPOX: --install-scripts is another distutils option which instructs it to install files listed under scripts=[] (seen it in setup.py) to a given directory.20:59
DktrKranz< rmunn|lernid> QUESTION: Won't DktrKranz's approach fail once Python 2.7 is released and absolute imports (http://www.python.org/dev/peps/pep-0328/)21:00
DktrKranzbecome the default? Isn't it better to fiddle with sys.path?21:00
DktrKranzrmunn|lernid: nice question. I haven't had the occasion to look at relative and absolute imports yet applied to Debian packaging, but I guess something have to be adjusted for that, and sys.path mangling could become handy.21:02
DktrKranz< mhall119|work> QUESTION: is rules.cdbs being used in what we're currently doing?  I see that your patches updated it as well21:03
DktrKranzmhall119|work: not directly. I included it in my example for those familiar with CDBS over dh7. I patched it to show how to handle things with different helpers.21:03
DktrKranz(debian/rules.olddh follows the same rule)21:04
DktrKranz< mhall119|work> QUESTION: is setup.py being called by anything in this setup?21:06
DktrKranzmhall119|work: it's called by dh_auto_install21:06
DktrKranzwhich in turn is called by dh sequencer, yay for tiny rules :)21:07
DktrKranz< Omar871> QUESTION: Which channel can we find you at?21:08
DktrKranzOmar871: several maintainer and developers interested in Python packaging are on #debian-python, OFTC network21:09
DktrKranzfeel free to join, ask questions and look for sponsorship :)21:09
DktrKranz< Emilien> QUESTION: elaboration: Well, I've heard that with Quickly you can package your app, which is written in Python too... Wouldn't it be easier21:12
DktrKranz                 to use that instead of having to set everything manually up?21:12
DktrKranzEmilien: I've heard of Quicly, I know is active developed by some cool guys here, I didn't have the occasion to test it yet, I still prefer using my favourite editor, just because things to cut are huge.21:13
DktrKranzso, I can't answer to that question, but I guess if it manages things as described here today, it should be fine (but please don't completely trust helpers)21:14
DktrKranz< Emilien> dktrkranz: what do you mean by "things to cut are huge"?21:15
DktrKranzEmilien: usually, templates are prepared to fit everyone's needs, while I prefer to tailor a package to fit its needs, removing every unneeded bit.21:16
DktrKranzOK, we're a bit late, if you still have questions, we can move to our usual channels. Thanks everyone for coming, I had a great time! See you :)21:19
=== lucio is now known as Guest6501

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!