/srv/irclogs.ubuntu.com/2012/03/07/#bzr.txt

=== zyga-xchat is now known as zyga
mgzmorning all09:12
jelmeryo yo yo09:13
vilahelli hello09:13
LarstiQmoin11:42
mgzhey LarstiQ11:45
LarstiQhey mgz :)11:49
=== zyga is now known as zyga-afk
=== zyga-afk is now known as zyga
jelmermgz, vila: I'm taking a long lunch break today, and will work a bit into the evening13:10
mgzwhoops13:35
vilajelmer: enjoy !13:54
jam1vila, jelmer: I was just getting a traceback trying to do "bzr lp-open lp:...", is that a known bug?14:09
jam1(on Windows, it gives an error about the certificate path)14:09
=== jam1 is now known as jam
vilabzr version ?14:10
vilaand running from sources or installer ?14:10
jamvila: sources, bzr.dev 644614:11
jamvila: http://paste.ubuntu.com/873049/14:11
jamit is true that /etc/ssl/... doesn't exist, but I wouldn't expect it to on Windows.14:12
vilajam: try again with trunk14:12
vilajam: reading the release-notes should also give you most of the details14:13
jamvila: I was just getting a traceback, and wanted to check if it was known before reporting it.14:14
abentleyjelmer: I have a merge request for bzr-pqm up that fixes daily builds.  Do you mind reviewing it?  Should I be asking gz?  https://code.launchpad.net/~abentley/bzr-pqm/config-stack-fixes/+merge/9623314:14
vilajam: k, several issues related to making urllib the default https implementation has been fixed14:15
jamvila: well, I've never installed pycurl :)14:15
jambut sure14:15
jamI just upgraded and get:14:15
jam$ bzr lp-open lp:~jameinel/u1db/c-pysync14:15
jamNot checking SSL certificate for xmlrpc.launchpad.net: 44314:15
jamOpening https://code.launchpad.net/~jameinel/u1db/c-pysync in web browser14:15
jamso it looks like it at least handles not finding the certificates14:15
vilajam: well, my remark also implied actually verifying the ssl certs14:16
vilajam: I don't remember the details but I think it doesn't verify on windows if it can't find the ca certs14:16
vilajam: i.e. now you know you're insecure :)14:17
jamvila: sure. It never used to verify them, which if it can't, it can't. The bigger thing is not giving a huge traceback one way or another. :)14:17
jamI'm not particularly worried about a MitM collecting my xmlrpc lookups to launchpad to expand lp: urls.14:18
vilajam: yup, lack of feedback from people running from sources on windows is to blame here I think ;)14:18
jamvila: well, I'm running source all the time, just not doing "bzr up" quite as often.14:19
jamand certainly, I rarely do "bzr lp-open lp:.." because usually "bzr lp-open" without arguments doesn what I want.14:19
jamand *that* doesn't fail, because I have public_branch = http:... set.14:19
vilahaaaa14:19
vilaI was wondering how you could have avoided the issue for so long ;)14:19
vilajam: it means even an heavy user like you doesn't use https that much... interesting14:21
jamvila: I use https almost constantly14:21
jamit wasn't failing for a lot of other cases14:21
jambecause we do client side resolution for most things14:22
jamapparently lp-open doesn't14:22
vilajam: well, you *don't* use it that much or you would have encounter the issue sooner14:22
jamvila: well, I use https, I apparently don't use xmlrpc+https via bzr14:22
vilahmm, weird, I don't remember specific fixes for xmlrpc...14:23
vilaand  your traceback is clearly in urllib14:24
jamvila: urllib as triggered from xmlrpc14:25
jambut sure, I push/pull via ssh14:25
jamso I don't use bzr transport over https for much of anything14:25
* vila nods14:25
vilaexactly why I found it interesting, most of the traffic for lp goes to the smart server these days so https is less used than I thought14:27
jamvila: well certainly whatever doesn't go to the smart server would end up going to plain http and not https14:28
jamand has been that way for a *long* time14:28
jamI do believe https://bazaar.launchpad.net exists now, *but* it only servers Loggerhead, and not the static files, IIRC14:28
jamyou can go to: https://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/changes or http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/changes and get the same content14:29
jambut you can only go to http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/.bzr/repository/format14:29
jamhttps://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/.bzr/repository/format is a 40414:29
abentleymgz: Could you please review https://code.launchpad.net/~abentley/bzr-pqm/config-stack-fixes/+merge/96233 ?15:06
santagadaanyone has an idea why I keep getting bzr killed when trying to branch lp:openobject-addons/6.1 on ec2?15:17
santagadait is a huge repo but I don't see why this happens15:18
=== mrevell_ is now known as mrevell
mgzabentley: sure15:38
mgzsantagada: OOM?15:42
LarstiQsantagada: do you have access to syslog?15:42
LarstiQvila: heya, can you see if there might be backlogged mail for me from the list, or should I ask a sysadmin?15:43
mgzvila is in the set of admins so he should be able to see (jelmer and I are not though)15:45
* LarstiQ nods15:48
vilaLarstiQ: let me look15:48
vila. o O (Which of the 16423 unread mails in the 104 mailboxes will give me the right url...)15:49
vilaLarstiQ: bazaar ml ?15:49
LarstiQvila: https://lists.canonical.com/mailman/admin/bazaar ?15:49
LarstiQvila: ueah15:49
LarstiQvila: I had to rush moving my vm, and had a missing mx for too long15:50
vilaLarstiQ: nope, nothing from you15:51
vilaerrr, wait, all I can see there is mail blocked for unsubscribed people, that's not what you're after right ?15:52
LarstiQvila: no, I'd be something like 'bounces' or undeliverable email15:53
* LarstiQ tries to look at a mailman interface15:53
vilamailman/admin/bazaar/members?letter=l says you're still subscribed15:55
LarstiQvila: ok, that's good15:55
vilawith your dyndns.org address that is15:55
LarstiQvila: yeah. That is why it went wrong ;). I'm trying to retire it this month15:56
LarstiQok, so now just need to figure out why I can't even get a password reminder delivered15:56
santagadaLarstiQ, yep15:57
santagadamgz, good catch15:58
vilaand the bounce settings seems sane: Mailman notify you, the list owner, when bounces cause a member's subscription to be disabled? (Yes) and Mailman notify you, the list owner, when bounces cause a member to be unsubscribed? (Yes)15:58
LarstiQgood15:58
LarstiQvila: thanks!15:58
santagadayep, the oom is killing the process15:59
santagadais there any way to tell bzr to not use 600mb of ram?15:59
santagadabzr 2.4.1 here15:59
santagadais bzr 2.5b6 better in this regard?16:00
santagadaand the homepage of bzaar is pointing to the old blog... last post is about the move16:01
mgzsantagada: depends how the memory is being used16:14
mgzwhat's near the end of .bzr.log before getting killed?16:15
santagadamgz, where is this file?16:15
mgz`bzr version` will tell you16:15
mgzit's perfectly possible for a big branch to use 600mb and repacking tends to trigger maximum usage16:17
santagadamgz,  456.915  24 bytes left on the HTTP socket16:17
santagadamgz, so I guess... it was downloading it still16:17
mgz...I did mean a bit more context, but yeah, that looks like it was during fetching16:18
mgzso, try branching not over http would be tip #116:18
vila... and jelmer and I fixed some issues with http buffering (but was it in 2.5 or on trunk...)16:18
mgzso, bzr+ssh if possible16:19
* vila nods16:19
vilabzr+ssh should behave better16:19
santagadabut it shouldn't be doing that right?16:19
mgzthe way dumb transports work though, you still end up loading large chunks of remote packfiles and doing operations, which is bad for really big branches16:19
santagadausing 600mb of memory just for buffering a download16:19
santagadawill try bzr+ssh16:20
mgzit's more the loading large chunk of pack, decompressing it, and trying to do stuff that adds up16:20
santagadaI thought http was not a dumb transport anymore...16:20
mgzpeople have in the past had problems with a single mistaken revision, present in the repo but actually removed from the branch,16:21
mgzthat was large but compressed really well,16:21
santagadato use ssh I need to do bzr launchpad-login, and that is it?16:22
mgzand http is dumb enough that if it happens to fetch that section of the packfile because parts either side are needed,16:22
mgzit basically dies to the decompression bomb16:22
mgzsantagada: yup, provided you have a local ssh key registered with launchpad16:22
santagadaisn't there a way to do anonymous ssh?16:22
santagadaI know how ridiculous this sounds, but is just a read16:23
mgzmaking people have ssh setup is a pain, particularly for non devs, but it then makes a bunch of other sticky problems just work16:24
santagadamgz, it is a server machine that is only downloading code... so having a ssh key there is kind of a security problem16:25
santagadaI will create a fake launchpad login then16:25
santagada:)16:25
santagadacould be called anonymous or download-only16:25
wgzit doesn't need to be *your* key16:27
santagadawgz, it needs to be a key that is attached to a user right?16:28
wgzthough, the account you register it with does then give write access to that key for all the projects the account has access to16:28
santagadawgz, so I should create my own anonymous account for it to work :)16:29
wgzbut for instance, ~bzr-pqm is not a human but has a key16:29
santagadasorry for being a d*ck. it's just that launchpad+bzr drives me mad from time to time16:30
santagadalike, why the hell this repo is 500mb16:30
wgzthat too is probably a good question :)16:31
santagadaI would bet on a mixture of bzr repos being big and human error16:31
wgzthere's certainly a culture of "just stick everything in svn" that some people have carried over to bzr16:32
wgzwhich... can be a bad thing16:32
santagadawell the repo has 150 mb of text in it. still I don't think the repo for some years should be 500mb16:33
vilalast time people did measurements bzr repos sizes were in the same range than git16:34
santagadavila, I will try to convert this one to git16:35
vilasantagada: let us know the results16:38
jelmerabentley: sure, I can have a look oif mgz hasn't yet16:43
abentleyjelmer: he got it, but thanks.16:43
wgzfeel free to look anyway :)16:44
santagadavila, its going to take 40 minutes... but let's see16:47
=== deryck is now known as deryck[lunch]
wgzjelmer, for bug 94909317:10
ubot5Launchpad bug 949093 in Bazaar "bazaar and subversion clashed" [Undecided,New] https://launchpad.net/bugs/94909317:10
wgzwait, no, misread, he does have his own sv libraries installed17:12
wgz*snv17:12
wgz...17:12
wgz*svn17:12
wgzso what we ship with the all-in-one installer is new enough?17:12
sledgeshello17:18
sledgesis there an equivalent of a git commit hash in launchpad, so that e.g. linaro kernel binaries would contain a string somewhere in a binary pointing to an exact state of the source it was compiled from?17:18
=== zyga is now known as zyga-food
santagadaisn't bzr+ssh supposed to be faster? It is both slower in terms of speed on launchpad (10x slower) and is transfering the same ammount of data17:28
santagadawgz, vila: the process got killed again, was downloading the repo from bzr+ssh and used all the memory again17:37
santagadalast line on the log "550.219  fetching: <SearchResult search:(set(['odo@openerp.com-20120307095627-8e2pf7955wg0jsoo']), ['hmo@tinyerp.com-20081124140910-4mqqv4qzd2rasrst', 'sma@tinyerp.com-20090924065200-nwj675pes1mole3d', 'fp@tinyerp.com-20090122131004-s39c7drfwbm8kggw', 'olt@tinyerp.com-20090126152714-d24104dspg0pyi8h', 'stephane@tinyerp.com-20090130202409-7dp454o39h2lo2pf', ...], 32283)>"17:38
santagadaany other ideas on how to make it not use <repo size> ram when branching?17:45
mgzsledges: depending on what you want, the revid is an option17:45
mgzsantagada: there are various cases where bzr+ssh isn't quicker, mostly because both sides do CPU work when splatting bytes is sometimes enough17:46
mgzsantagada: try `bzr branch -r100 BRANCH` then `bzr pull -r200` and so on17:47
sledgesmgz, I have kernel image "Ubuntu 2.6.38-1401.4-linaro-lt-mx5 2.6.38.8" from official ubuntu-11.10-preinstalled-desktop-armel+mx5.img.gz booting fine on i.MX53QSB, but its source compiled on my HOST halt after "Uncompressing kernel..." (same .config, and only minor gcc version difference) ? (source took from:  http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/precise/linux-linaro-lt-mx5/precise/revision/317:47
santagadamgz, yep bzr+ssh seems to be transfering the same ammount of data, and ssh in launchpad ssems to be 10x slower than http17:47
santagadamgz, what does bzr branch -r100 do?17:47
mgzgets the first 100 revs.17:48
santagadaseriously that bzr don't do any file buffering and I will have to do it manually?17:48
mgzsledges: I'm not following... there are lots of reasons compiling the same stuff locally could break for you17:48
mgzare you trying to submit a bug report with enough details for someone else to repo, or change the current build process in linaro?17:49
mgzsantagada: it mostly helps narrowing down specific problem revisions17:50
sledgesmgz, I'm not asking you to fix my build problems, I am explaining why I need the revid you now mentioned; now how can I fish it out from the only binary I have..17:50
mgzclearly doing it manually isn't more efficient that what bzr does internally17:50
sledgesin #linaro I've been told that launchpad source state revision might not be the one that was put into ubuntu preinstalled img17:50
mgzsledges: you'd need to ask the linaro guys how they tag their kernels, I'm not involved with that17:50
santagadamgz, bzr is using ram to fetch the repository... I don't think it is one particular revision that makes it use 600mb of ram17:51
mgzunless their build process explicitly embeds some unique indentifier, there's no reason to suspect you could pull one out of their binary17:51
sledgesmgz, Git has the hash tag for every commit, I came here to ask you whether there is an equivalent (see my original question)17:52
mgzwhat gets put in the binary is part of the build process, not the version control system though17:52
mgznothing in bzr puts revision identifiers in other people's makefiles17:52
sledgesthank you, that answers my question17:53
mgzit may well be done as a matter of course, but it's not something I'm aware of how to retrieve for projects I don't know the details of17:53
santagadamgz, bzr is downloading all revisions even if I say bzr branch -r 10017:54
santagadaahh it was downloading everything because I went back to use http. now on ssh it did branch only the first n revs17:55
mgzsantagada: without actually debugging it in detail, all I can do is offer suggestions, there are methods of getting exactly what is held in memory, but the reaper gets in the way of that17:56
mgz100 revs at a time sucks, but does at least rule out certain classes of problem17:56
santagadamgz, I will do 10000 at a time now17:57
mgz...dear god, how big is this thing? :)17:57
santagada40k revs17:57
mgzehehe17:57
santagadaits a 500mb repo17:57
mgzmost 500mb repos are just from people versioning huge trees17:58
mgzthe emacs-class ones that actually have decades of history are rarer17:58
santagadathis is the case17:58
santagada150mb of text files of tons of different things17:58
mgzanyway, it's probably worth filing a bug/posting to the mailing list about your woes today anyway17:59
santagadaand translated to tons of languages using the launchpad tool17:59
mgzit's good to have a record of what problems people run into in practice17:59
santagadamgz, I don't feel like doing that when you see bugs like this marked as wont fix17:59
santagadamgz, https://bugs.launchpad.net/launchpad/+bug/49338918:00
ubot5Ubuntu bug 493389 in Launchpad itself "launchpad should offer anonymous bzr+ssh" [Undecided,Won't fix]18:00
mgzwell, won't-fix would not be the case here, but a short post about with the rev stats and what you tried and how it failed is probably of more use18:00
mgzas fuzzy bugs are less useful than just shared knowledge about where problems still exist18:01
santagadaI still think it is not a fuzzy bug, bzr seems to be loading the whole repo in memory while fetching18:01
mgzI can believe we have room to improve general memory usage for trees as deep as yours still, and people may have more insight18:02
mgz"whole repo in memory" isn't really a diagnosis, as that's not what bzr actually does18:03
mgzit probably is holding the entire revision graph in memory, and caching chunks of packfile in uncompressed form18:04
mgzand probably some other things too18:04
mgzbut that's not the same as just reading all the compressed bytes from disk and leaving them referenced18:04
mgzright, I started this VM half an hour ago, I really should start using it...18:05
santagadamgz, so why doesn't it first download the whole repo to .bzr and only then start building the revision graph/doing stuff with the data?18:08
santagadathe fetch phase should use very little ram18:08
santagadaI was able to get it using bzr branch --stacked lp:openobject-addons/6.1 addons18:09
mgzideally fetching would just splat bytes over the network18:09
santagadabut only using bzr+ssh, using http it downloads the whole repo anyway18:09
santagadamgz, exactly18:10
mgzthe reason it doesn't is it's doing lots of work for the other common case, where you only want a subset of revisions from the repo18:10
mgzwhich is generally the case for updates, and with launchpad and stacking, is alwasy true even for initial fetches18:10
mgzstacking on a remote branch is probably not what you want by the way.18:14
santagada--stacked proved to be a really bad idea when you need to do anything with the repo later as everything is done on the network18:14
* mgz won that race here :)18:14
=== zyga-food is now known as zyga
santagadaso --stacked is a big no on development. but for this particular case (deployment) its ok I guess18:15
santagadaI will only be doing bzr update here18:15
mgzright, as would just export be perhaps?18:16
santagadabut really bzr branch should be much smarter18:16
mgzrather than stacking, you could just do a checkout18:16
mgzwhich is a bit less voodoo.18:16
santagadamgz, what is the difference between branch --stacked and checkout?18:16
mgzthough... both those want the 2.5 fixes on launchpad18:16
santagadamgz, I thought the only diff between branch and checkout is that checkout make the branch bounded to the parent18:17
mgza (lightweight) checkout knows the branch lives somewhere else, a stacked branch just falls back to a different location if it can't find a revision locally18:17
santagadaexport would not work because we would generally want a faster deploy, and doing a pull+update is usually faster than a whole repo copy18:18
mgzbut as said, both have pretty bad network behaviour, some of which is fixed in bzr 2.518:18
=== deryck[lunch] is now known as deryck
santagadamgz, in practice what is the difference?18:18
santagadathat description seemed to mean the same18:19
mgza checkout has consistent semantics and performance18:19
santagadaso it always look on the network, and stacked sometimes do?18:19
mgz(currently consistently bad with 2.4 though)18:19
santagadaI will take stacked any day of the week for anything :)18:20
santagadamgz, I'm wondering, which would you use on the deployment use case?18:21
mgzfor your case you probably do just want a branch copy, so maybe stacked is sort-of-mostly okay, it's historically been buggier though18:22
mgz^hm.18:22
mgzrsync probably :)18:23
mgzI'm not sure, there are mixed priorities on deploying from vcs and the things I've needed to do it for have been small enough that it mostly hasn't mattered.18:24
mgzI think there's been some past discussion on the mailing list on the topic.18:25
mgzI'll see what I can dig up once I get this build started18:27
mgz...I just got email from github about checking ssh keys after the compromise18:29
mgzam failing to pin the keyword on the donkey18:37
mgzthought "deployment" would pull up relevent threads... what other terms are used...18:37
kkrevI ran cygwin update and am getting this all the sudden: ERROR: Don't know how to handle SSH connections. Please set BZR_SSH environment variable.18:38
mgzif you run `ssh -V` in your cygwin shell what do you get?18:40
kkrevActually I think the issue is something else. I get that on pull, but branching from the ssh hosted repository works fine. This is the pull error: 8 [main] python2.6 1412 child_info_fork::abort: unable to remap _chk_map_pyx.dll to same address as parent (00B70000) - try running rebaseall18:41
kkrevthere is no rebaseall command18:41
mgzthat's... not a good sign18:42
mgzso, as a workaround you can probably move the compiled extention out of the way18:43
mgzbut the cygwin packagers would probably appreciate a bug report18:43
mgzsome cygwin internal machinery is going wrong with one of our (optional) compiled extensions basically18:44
mgzcould be a bug with cygwin's C runtime or with cython, is unlikely to be a bzr issue though18:44
kkrevthe error wasn't lying about running 'rebaseall'. that's a cygwin thing. http://www.heikkitoivonen.net/blog/2008/11/26/cygwin-upgrades-and-rebaseall/18:55
kkrevit works fine after that. I was just confused by assuming rebaseall was a bzr thing.18:56
mgzah, so it's upgrade specific?18:56
mgzI guess they don't have a good mechanism for updating older packages when they change their core dll18:56
kkrevright. I had to reboot the machine and run that rebaseall command. no idea exactly what it does, but I assume it updates some dll registry.18:57
mgzwell, as long as it works now :)18:59
=== yofel_ is now known as yofel
mgrandii'm having trouble with locks and pushing to github, its worked before, but now it seems to be creating a lock and then saying its locked by someone else, when that person is myself21:34
mgrandihttp://bpaste.net/show/24741/21:35
mgrandirelevant info in the log file: http://bpaste.net/show/24742/21:36
mgrandii need to push this code to github >.>21:56
pooliehi all22:13
mgrandihey22:17
mgrandidid you see what i posted?22:17
mgrandiabove?22:17
pooliemgrandi, no22:25
mgrandii'm having trouble with locks and pushing to github, its worked before, but now it seems to be creating a lock and then saying its locked by someone else, when that person is myself22:25
mgrandi14:35 mgrandi22:25
mgrandihttp://bpaste.net/show/24741/22:25
mgrandi14:36 mgrandi22:25
mgrandirelevant info in the log file: http://bpaste.net/show/24742/22:25
pooliehm22:26
pooliethis sounds familiar22:26
poolieare you on 2.5.0?22:26
pooliei guess, then, file a bug against bzr-git22:26
mgrandi2.5b622:26
mgrandiok, any idea on how to get this work temporarily? >.>22:29
* jelmer waves22:32
mgrandibzr-git got updated recently, maybe i'll try updating that22:33
jelmermgrandi: hmm, that's odd22:33
mgrandiyeah. it worked fine before22:33
mgrandibut then i did it today, it repacked the repo22:33
mgrandiand now it seems to be repacking it when it doesn't need it, and then gets caught in the lock condition22:34
jelmermgrandi: I'll have a look when I get back home22:37
mgrandiok. i should be around, ask if you have any questions22:37
mgrandii am trying with the latest version of bazaar and bzr-git22:38
mgrandiannnd same thing22:40
spivOnly if he wants to give the tute sober.22:42
spivOops, wrong window!22:42
* spiv quickly reads scrollback to see if that might have made accidental sense22:43
mgrandino22:43
mgrandiyou didnt =P22:43
* jelmer tries to make sense of the word 'tute'22:43
spivShort for "tutorial"22:44
mgrandihow is it short if tutorial has no e in it haha22:44
spivBecause the magic abbreviation pixies declare it to be so!22:45
mgrandibah. got me there.22:45
mgrandianyway, jelmer, any idea on how to get around this temporarily? i kinda want to push to github real fast >.>22:45
jelmermgrandi: I would try reverting back to an older version of bzr-git temporarily22:46
jelmerI'll have a look at the issue when I get back home, which should be in anbout 30 min22:46
mgrandiah22:46
* jelmer is in a train with a attery that is about to run out22:46
jelmermy laptop;22:46
jelmer's battery that is, not the train's22:46
mgrandii shall be here!22:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!