/srv/irclogs.ubuntu.com/2015/06/12/#ubuntu-ci-eng.txt

=== _salem is now known as salem_
=== salem_ is now known as _salem
michiyou05:12
=== chihchun_afk is now known as chihchun
robruMe?06:56
sil2100jibel: hey! Did alesage finish testing image number 10?07:19
pstolowskitrainguards, jibel hello! i'm not sure what happened to line #22 in the cisheet, looking at the dashboard looks like it landed?07:28
Mirvpstolowski: the spreadsheet often loses the status after landing happened07:29
jibelsil2100, ToyKeeper did it but she couldn't test MMS on affected networks (giffgaff and vodafone)07:29
jibelsil2100, everything else is OK07:29
pstolowskiMirv, ah, ok, cool then, thanks07:29
jibelpstolowski, what was line 22?07:29
jibelgood morning :)07:30
pstolowskijibel, Review editing (Fix: LP #1318144) for vivid ppa-overlay, don't remember what was the silo07:30
ubot5Launchpad bug 1318144 in unity-scope-click (Ubuntu) "Can't edit reviews" [High,In progress] https://launchpad.net/bugs/131814407:30
Mirvjibel: silo 02907:30
jibelpstolowski, yes it landed in silo 02907:31
jibelas Mirv said07:31
jibelpstolowski, rvr did the verification yesterday07:31
sil2100jibel: you think I can copy it over to the RC channel?07:31
pstolowskijibel, great, ty07:31
sil2100jibel: or do you want davmor2 to run tests before I do that?07:31
sil2100jibel: anyway, I can finally feel the release clos07:32
sil2100*close07:32
jibelsil2100, I'd like davmor2 to confirm it's good to go07:32
jibelsil2100, he can test on giffgaff07:33
jibelsil2100, what is the story with shell rotation? silo 18 landed and silo 4 must be rebuilt now?07:34
sil2100jibel: yes, but I'm still not sure if that will help... let me try07:35
jibelsil2100, apparently the is a build started 5h ago by kgunn https://ci-train.ubuntu.com/job/ubuntu-landing-004-1-build/202/ do you know what is going on?07:43
jibelthere is*07:43
sil2100Ok, now at least it's clear why it doesn't build, let me dive into that07:53
sil2100jibel: anyway, looks like silo 4 will be ready soon :)08:14
mzanettisil2100, don't think so...08:31
mzanettisil2100, the job was started 5h ago08:32
mzanettiseems to hang :(08:32
sil2100mzanetti: now the i386 in wily fails08:32
sil2100mzanetti: I thought that was transient, but cmake seems to die08:32
mzanettiyeah.. saw that yesterday08:32
mzanettithis one right now is again different tho08:32
mzanettiseems to be locked08:32
sil2100mzanetti: yeah, so...08:34
sil2100mzanetti: Mirv said cmake has some issues on i386 on wily now08:34
sil2100So it's unrelated, we need to wait for that to get fixed :D08:34
mzanettiI'm so sure something else will break in the meantime08:37
Mirvmzanetti: that's my thoughts with wily atm, trying to land qt 5.4.2 before my holidays in a week, and things just keep on breaking :)08:59
Mirvbut now I think if it's all down to ciborium / Go / OpenGL on armhf08:59
Mirvbug #1464569 about the CMake issue09:01
ubot5bug 1464569 in gnutls28 (Ubuntu) "3.3.15-5ubuntu1 breaks CMake on i386" [Undecided,New] https://launchpad.net/bugs/146456909:01
john-mcaleelysil2100, so, what's the ota-4 status?09:04
=== tvoss is now known as tvoss|test
=== tvoss|test is now known as tvoss
sil2100john-mcaleely: ...almost there! (you're probably annoyed by this already)09:15
sil2100john-mcaleely: so, ToyKeeper did some tests on the candidate yesterday, now davmor2 confirms a fix that reqired giffgaff and we'll copy to RC09:15
sil2100It should be copied to RC really really soon09:15
sil2100Like, minutish-soon09:15
john-mcaleelysil2100, not annoyed at all. fatalistic :-)09:16
john-mcaleelyand rc soon sounds excellent09:17
sil2100mzanetti: would you be fine with rebuilding unity8 in the PPA? Since I think I'll disable -proposed in it and get unity8 building finally09:20
davmor2sil2100: jibel: mms works in both directions on giffgaff and ee09:33
davmor2sil2100: also mms group is off09:33
davmor2john-mcaleely: ^09:38
john-mcaleely\o/09:38
davmor2sil2100: have you pushed it yet?09:38
davmor2sil2100: is it nearly there yet?09:38
davmor2sil2100: is it nearly there yet?09:38
davmor2sil2100: is it nearly there yet?09:38
davmor2sil2100: is it nearly there yet?09:39
davmor2sil2100: is it nearly there yet?09:39
sil2100Pushing!09:39
sil2100Daaamn09:39
davmor2\o/09:39
davmor2oh oh09:39
john-mcaleelyDaamn doesn't sound good09:39
davmor2sil2100: john-mcaleely ah no vpn09:39
sil2100hm?09:40
ogra_yeah, seems VPN and IRC are down09:42
pedronisyes09:42
sil2100jibel, john-mcaleely, davmor2: krillin image 10 copying to rc right now09:43
john-mcaleely10?09:43
john-mcaleelythis all gets very confusing :-)09:43
ogra_john-mcaleely, just wait til we get rid of system-image :)09:44
sil2100Yeah, image 10 from the -factory-proposed channel is what we want ;)09:44
ogra_(there wont be image numbers anymore then)09:44
john-mcaleelylol09:44
sil2100jibel, john-mcaleely, davmor2: besides that I'll also promote the vegetahd, arale, generic and mako images09:45
ogra_(only versions of the different snaps)09:45
john-mcaleelyarale? not sure about that09:45
sil2100Ok, I thought we wanted to release everything at once09:45
john-mcaleelyie, changes in stable/meizu.en might need to be separately managed09:45
john-mcaleelyI think so too, but there's a discussion in my inbox this morning about thta09:46
john-mcaleelythat will head your way shortly :-)09:46
sil2100Anyway, the -factory-proposed channel also has a correct arale image based off the snapshot PPA + the right device and custom tarballs09:46
john-mcaleelyperfect09:46
sil2100jibel, john-mcaleely: ok, both krillin and vegetahd are on rc now10:33
john-mcaleelysil2100, thank you. shame my build server is down with the golden factory bits on it10:34
john-mcaleelyhopefully I have a local copy10:34
john-mcaleely(looking now)10:34
sil2100davmor2, jibel: can you guys do the OTA testing?10:35
sil2100:)10:35
sil2100john-mcaleely: don't send it to BQ just yet tho, we just want to confirm OTA works fine10:36
davmor2sil2100: this is in the standard rc channel right?10:36
john-mcaleelysil2100, ack10:36
sil2100davmor2: yeah, the ubuntu-touch/rc/bq-aquaris.en10:36
davmor2sil2100: sure on it10:36
sil2100\o/10:36
* sil2100 copies over the generic image10:37
bzoltansil2100: Could you please throw me a silo? :)10:44
sil2100Suar10:45
sil2100Throwing10:46
sil2100;)10:46
bzoltansil2100: thanks ...10:49
* bzoltan catches it10:49
davmor2bzoltan: fumbles.....recovers......catches it with his foot10:53
mzanettisil2100, do we know why the builders are just stuck?10:56
sil2100mzanetti: didn't get an answer from you on my last question I think :)10:57
mzanettisil2100, I had serious troubles with connectivity in the past hour10:57
sil2100mzanetti: would it be fine to rebuild all unity8 archs in silo 4?10:57
mzanettisorry10:57
mzanettisil2100, yeah, works for me10:58
sil2100bzoltan: hmmm10:58
sil2100bzoltan: not sure why, but the prepare-silo job is stuck :O10:58
sil2100So you didn't get the silo yet10:59
sil2100I have no idea what's going on with everythin today10:59
sil2100mzanetti: hey, did you press rebuild?11:00
mzanettisil2100, yeah, like some 15 minutes back11:00
mzanettibefore that last chat we had11:00
mzanettinot gonna touch it now11:01
sil2100That's too early, I didn't disable -proposed, but yeah... it's stuck anyway so no worries11:01
sil2100I think the jenkins is dead somehow11:01
mzanettisil2100, can you try to get it built, I'd go for an early lunch then11:01
mzanettito be back when it's build so I can do the testing asap11:01
sil2100Yeah... but our train jenkins seems hm, dead11:02
mzanettiwouldn't the build happen in the ppa anyways? I thought you'd upload it manually...11:02
mzanettibut anyhow... whatever works...11:02
mzanettior not works :D11:02
sil2100Well, jenkins needs to prepare the packages first ;p11:02
mzanettiah, I see11:03
davmor2jibel, john-mcaleely, sil2100: image 16 in place, backup installed, waiting on ota to download now11:05
sil2100\o/11:05
john-mcaleelyooh11:05
davmor2full image download 427.7mb11:07
Mirvsil2100: you're saying my next obstacle today will be that train is practically dead? :D11:08
sil2100;/11:08
sil2100I'm poking webops, not sure where the problem is11:08
Mirvsil2100: I've gotten cmake, go and UITK broken for me today so far11:08
davmor2Mirv: think of it as training in futility11:08
Mirvdavmor2: or sisu11:09
sil2100Mirv: do you have any spare silo you could run the build watch-only job with debugging?11:09
Mirvoh there's an article about it in Wikipedia https://en.wikipedia.org/wiki/Sisu11:09
Mirvsil2100: feel free to abort https://ci-train.ubuntu.com/job/ubuntu-landing-016-1-build/174/console and fire it again with "ciborium" in pacakges but with debug.11:10
sil2100Mirv: hm, I don't like breaking the state in the middle of that, but ok :) Thanks!11:10
sil2100Mirv: looks like SSO issues probably11:11
Mirvsil2100: it's been there for 15 minutes doing nothing..11:11
nik90davmor2: full image download 427.7mb! Full of awesome goodies ;) ...lets hope we get it by monday11:11
Mirvhaha, "See also: Cojones"11:11
sil2100hm, SSO for users seems to work, but CI Train doesn't seem able to authorize11:13
sil2100At least that's what I think is happening11:13
sil2100mzanetti: looks like the world doesn't want silo 4 to land11:14
davmor2sil2100: image build 24/ubuntu 15.04 -armhf (20150611-173952) so that looks right :)11:18
bzoltansil2100: and the world does not want a silo for the UITK either :)11:18
sil2100;/11:18
sil2100wth is going on11:19
jibeldavmor2, hm, this 427MB will be a problem. It has to be downloaded completely to receive the notification, correct?11:20
davmor2jibel: indeed11:21
jibeldavmor2, if the connection drops during download, does it resume or restart from scratch?11:21
wgrantsil2100: When did the broken bits of the train last work?11:22
davmor2jibel: no idea.  Not sure if it currently registers the fact there is an update, I'll try that afterwards11:22
sil2100wgrant: not sure, would have to browse the history, not sure if anyone ran anything in the morning11:23
sil2100Mirv: did you run any CI Train code successfully?11:23
Mirvsil2100: I haven't tried anything else besides building now11:23
wgrantsil2100: I wouldn't have expected anything to break in the last 24 hours, but login.ubuntu.com moved to new IP addresses a couple of days ago, which could have broken firewall rules.11:23
sil2100hm, SSO works fine on my machine11:24
wgrantYes, SSO itself is fine.11:24
sil2100I wonder what can be happening, I don't have access to the train machine so I can't debug11:24
sil2100CI Train uses a credentials file when logging in, could that be a problem11:25
sil2100?11:25
wgrantWhen logging into what?11:25
wgrantSSO doesn't have the concept of a credentials file.11:25
wgrantsil2100: What is the exact symptom you're seeing?11:26
jibeldavmor2, can you test 22 (latest promoted image) to 24 too?11:26
davmor2jibel: sure11:27
sil2100wgrant: all of our CI Train jenkins build jobs that use LP just hang somewhere in the beginning, when aborting it looks as if it had problems authorizing11:28
wgrantsil2100: Hang where?11:28
sil2100https://ci-train.ubuntu.com/job/ubuntu-landing-016-1-build/174/console11:28
wgrantWhat's the message?11:28
wgrantOh, that doesn't looks like anything related to SSO.11:28
sil2100I have no idea, no message, it's just waiting on something and blocking11:28
wgrantSSO is login.launchpad.net or login.ubuntu.com11:28
wgrantThat just looks like your Launchpad OAuth token has expired or been revoked.11:28
sil2100There's no message if you don't abort11:28
sil2100hm, but why does it hang then and not just abort straight away?11:29
wgrantBecause launchpadlib was written by fools, mostly.11:29
sil2100uuuh!11:29
wgrantIt assumes it's running interactively.11:29
=== sil2100 changed the topic of #ubuntu-ci-eng to: Need a silo or CI Train support? ping trainguards | Need help with something else? ping cihelp | Train Dashboard: http://bit.ly/1mDv1FS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: CI Train outage! The CI Train Bot user token expired, waiting for IS/webops
davmor2jibel: test of 16 passes trying on 22 now11:41
jibeldavmor2, do it quick before pmcgowan tries to shoehorn another fix ;)11:48
davmor2jibel: installing now11:49
davmor268% downloaded11:49
=== alan_g is now known as alan_g|lunch
davmor2jibel: it resumes12:05
davmor2jibel: download image on 22 now same 427.7 MB12:06
sil2100Yeah, full delta12:10
sil2100I mean, full image12:10
sil2100uh,12:11
ogra_and 2h boot time ... if you have a popeyesque amount of apps installed :)12:12
=== MacSlow is now known as MacSlow|lunch
pmcgowanI think I am getting a bad rap12:15
Mirvsil2100: you want to land silo 4 still today, in case the train gets fixed, right? I mean, I just need to rebuild unity8, qtmir, qtmir-gles, qtubuntu, qtubuntu-gles in my Qt silo if you land 4 first12:18
Mirvwhich is ok, just needs to be known12:19
MirvI need an approval for my ciborium branch anyway12:19
sil2100Mirv: yes, I'd like to land 4 ASAP12:21
sil2100But there's no one to help us...12:21
davmor2pmcgowan: you only think that?12:21
pmcgowanhey12:22
sil2100davmor2: so how are the OTA tests?12:22
davmor2pmcgowan: I've heard rap dude you are aweful at it ;)12:22
pmcgowandavmor2, they call me mc-gowan12:22
ogra_lol12:24
davmor2sil2100: pmcgowan jibel john-mcaleely 22-24 installed looks good https://docs.google.com/spreadsheets/d/1j91dnYhnffw_6BQCebNncXl4lcS6HMo-qVCdn_YdGmY/edit#gid=1940054 is for 16 → 24 I'll do the same for 22→24 after relocate12:26
davmor2and with that I am out of here12:26
Mirvsil2100: ok. and yes, no help yet :(12:27
sil2100eh, I have no juju knowledge and they request some info about that12:28
sil2100I never had time to dig into that12:28
dobeycihelp: there seems to be a problem with jenkins builders now? i tried to rebuild silo 22, but it never proceeds past the "INFO Including $packages" line https://ci-train.ubuntu.com/job/ubuntu-landing-022-1-build/99/console12:31
fgintherdobey, that's a question for trainguards ^12:32
dobeyfginther: oh, they own those jenkins jobs? i thought ci owned them?12:33
dobeyfginther: also, go back to sleep! :)12:34
fgintherdobey, yep, they look after ci-train12:34
jibeldobey, I think it's the same problem sil2100 was mentioning earlier about lp credentials for the citrain12:35
sil2100dobey: look at topic12:35
dobeyoh ok12:37
ogra_bot lost its train ticket12:37
dobeysil2100: ah, the topic is too long and trails off the end of my double-wide terminal, so unless topic changes while i'm looking at the channel (ie, not at my 02:00 AM :), i generally won't know to look at it. maybe would be good practice to start prepending important notices at the beginning of topic, rather than appending to the end, so they are more visible? :)12:40
sil2100Maybe, I'm not the owner of this channel ;)12:40
dobeytrainguards: could someone hit rebuild on lp for https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-022/+build/7526571 please?12:46
Mirvdobey: sure12:47
dobeyMirv: thanks12:47
=== Ursinha changed the topic of #ubuntu-ci-eng to: Need a silo or CI Train support? ping trainguards | Need help with something else? ping cihelp | Train Dashboard: http://bit.ly/1mDv1FS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: CI Train outage! This means CI Train jenkins builders are failing. The CI Train Bot user token expired, waiting for IS/webops.
mzanettisil2100, what happened?12:55
=== alan_g|lunch is now known as alan_g
=== joet is now known as josepht
=== _salem is now known as salem_
sil2100I can't publish that yeet!13:22
sil2100Aargh13:22
rvr:-/13:26
kgunnsil2100: so what's the deal atm ? (i've read all the scroll back...just wondering if we're doomed for the day)13:30
sil2100Webops is on it, there seems to be something wrong with the train instance13:31
ogra_well, there was quite an outage today13:32
ogra_(vpn and irc at least) ....13:32
ogra_probably fallout of that13:32
sil2100From what we were able to determine, with the migration to PS4.5 it seems that someone didn't deploy the train correctly and the credentials and keys were not set in stone13:33
sil2100So probably due to some outage those got erased or something13:33
sil2100...and I think jacekn resolved the issue13:35
sil2100Yaay!13:35
=== sil2100 changed the topic of #ubuntu-ci-eng to: Need a silo or CI Train support? ping trainguards | Need help with something else? ping cihelp | Train Dashboard: http://bit.ly/1mDv1FS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: there was a CI Train instance outage, everything should be back but inform us if you see any problems - builds need to be re-run
sil2100bzoltan: silo for you!13:37
sil2100dobey: train should be good now, just re-run the build job if it's stuck13:38
sil2100davmor2: how's the tests? Finished? :)13:38
dobeysil2100: ok, thanks13:38
sil2100davmor2: I'll have to AFK soon to get the cat, but if you finish the OTA tests with success then just poke john-mcaleely to push the image to BQ13:39
john-mcaleelysil2100,13:40
john-mcaleely<davmor2> sil2100: pmcgowan jibel john-mcaleely 22-24 installed looks good https://docs.google.com/spreadsheets/d/1j91dnYhnffw_6BQCebNncXl4lcS6HMo-qVCdn_YdGmY/edit#gid=1940054 is for 16 → 24 I'll do the same for 22→24 after relocate13:40
john-mcaleely<davmor2> and with that I am out of here13:40
john-mcaleelyis that done? ^ sil210013:41
sil2100I thought "I'll do the same for 22→24 after relocate" meant it's not done yet, but maybe I misunderstood13:42
sil2100If all is tested than yes, we're good :)13:42
john-mcaleelyah, yes13:42
sil2100jibel: ^ are we good?13:42
john-mcaleelyyou've parsed better than me13:42
john-mcaleelyI agree we're still waiting13:43
john-mcaleelysorry13:43
pmcgowanjohn-mcaleely, today fer sure!13:44
john-mcaleelypmcgowan, promise?13:45
john-mcaleely:-)13:45
jibelsil2100, john-mcaleely davmor2 did the upgrade from 22 to 24 and it "looks good" but he didn't report the results to the spreadsheet.13:47
john-mcaleelytoday fer sure then!13:47
=== MacSlow|lunch is now known as MacSlow
davmor2and back13:50
sil2100davmor2: are the OTA tests done? :)13:52
Mirvtrain build are still broken, thus that dput failure14:04
davmor2john-mcaleely: sil2100: there is nothing missing from 22→24 I'm just going to take the time to fill out the form now14:11
abeatotrainguards, can I get a silo for line 69?14:17
davmor2john-mcaleely, pmcgowan, sil2100, jibel: https://docs.google.com/spreadsheets/d/1j91dnYhnffw_6BQCebNncXl4lcS6HMo-qVCdn_YdGmY/edit#gid=938740349 22→24 is good too14:35
john-mcaleely\o/14:35
davmor2sorry lunch and relocate got in the way but thems the hits14:35
kgunnsil2100: thot i saw you say it was fixed, so tried a unity8 rebuild, failed dput in silo414:37
kgunn?14:37
charlesbdmurray, ted mentioned that the recoverable error API will be in the next release of libwhoopsie, dyk when that'll be?14:48
=== infinity1 is now known as infinity
charlesbah15:25
popeysil2100: do we have a date/time for OTA-4 yet? (sorry)16:27
sil2100popey: not sure, I think it's up to BQ now16:28
sil2100They should be done around Monday?16:28
popeyok16:28
Mirvsil2100: I realized that 004 doesn't have QA approval yet, so I guess it wouldn't land today anyway?16:34
jibelMirv, how would it have QA approval it didn't even build16:35
Mirvjibel: well it built for vivid just fine, and wily is not QA checked16:35
Mirvjibel: only wily would need rebuilds, vivid could stay as is16:35
MirvI'm just thinking about landing order and when my wily Qt could go in. UITK needs it so that they can begin their landing.16:36
jibelMirv, right but I think that's the point of dual landing otherwise it should land in 2 separate silos16:37
Mirvwell, hopefully 004 lands so I can land 016 on Monday16:54
=== chihchun is now known as chihchun_afk
=== alan_g is now known as alan_g|EOW
sil2100jibel, Mirv: silo 004 built17:23
sil2100mzanetti: is silo 004 good for QA? :)17:23
mzanettiuh... did it build?17:24
* mzanetti grabs a device and tests17:24
mzanettiwow! it did build17:24
mzanettiback in a bit... just doing sanity tests before asking QA to spend efforts17:24
sil2100Yeah, I got yelled on for that, but I hope at least this won't cause any breakage of the world17:25
mzanettikgunn, you just missed the conversation of the week :D17:25
sil2100Siiince I disabled -proposed for the time of the build17:25
sil2100:|17:25
tedgtrainguards, can someone rebuild this? Seems like it failed a signature check from the archive?!? https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-036/+build/753509717:40
sil2100cjwatson, wgrant: hey! Could you take a look at that one? ^17:40
sil2100Before we rebuild :)17:40
sil2100Ah17:40
sil2100cjwatson, wgrant: unping!17:40
sil2100cjwatson, wgrant: nevermind that17:41
sil2100tedg: ok, so this is the current breakage caused by an ongoing transition in proposed17:41
tedgOh, okay.17:41
sil2100tedg: https://bugs.launchpad.net/ubuntu/+source/gnutls28/+bug/146456917:41
ubot5Launchpad bug 1464569 in gnutls28 (Ubuntu) "3.3.15-5ubuntu1 breaks CMake on i386" [Undecided,Confirmed]17:41
=== tvoss is now known as tvoss|test
=== tvoss|test is now known as tvoss
sil2100This is our bug, but it's actually related to another one17:41
tedgsil2100, okay, so I need to wait for that to get fixed. But I can test the armhf binaries.17:43
cjwatsonThe signature check thing there is perfectly normal, just cosmetic.  We should silence it at some point; LP checks the signature much earlier, and individual buildds don't have the ability to do so.17:45
cjwatsontedg: ^-17:45
tedgAh, I see. It was the previous verbose message :-)17:46
=== Trevinho is now known as Trevinho|Holiday
tedgtrainguards, I marked line 45 as tested, because it is. But it shouldn't be published until the i386 binary can be rebuilt. I put a note to that effect in the spreadsheet.18:44
robrutedg: hm, can you not mark it as tested if it's not ready to publish. the dashboard will say "YOU CAN PUBLISH" in really big print and the warning not to publish is very small underneath that18:45
tedgrobru, K, changed it from "Yes" to "Yeah", does that take it off the dashboard?18:45
* tedg can do "Yup" too18:46
robrutedg: not sure, let me check18:46
robrutedg: what row are we talking about?18:47
tedgrobru, 4518:47
robruoh, row 45. i thought you said silo 45. duh18:47
robrutedg: oh, actually the "failed to build" status is great, that stops it from saying "ready to publish" anyway. so whatever. is somebody going to follow up that i386 build?18:48
tedgrobru, Yeah, it's being tracked. Apparently a lib and cmake crashing.18:48
tedgrobru, In the backlog18:49
robrutedg: ok, but I mean when somebody fixes that issue in the archive, is somebody going to remember to actually retry this particular build? because if not that'll just sit there ;-)18:49
tedgrobru, I'm not 100%, if not I'll be annoying people about it :-)18:50
robrutedg: ok great. yeah once it's good to go just ping me and I can publish it later.18:50
* robru -> lunch19:07
=== chihchun_afk is now known as chihchun
robrugood god20:51
=== ahayzen_ is now known as ahayzen
=== salem_ is now known as _salem

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!