/srv/irclogs.ubuntu.com/2010/05/24/#bzr.txt

igcmorning00:22
lifelesspoolie: thumper asked that we not use the queue feature for the moment00:24
lifelesshi igc00:24
igchi lifeless00:25
lifelesspoolie: 'e' for email is the new command in hydrazine00:25
poolieyeah00:26
poolieyou broke my muscle memory a bit there00:26
pooliehello igc!00:26
igchi poolie!00:26
lifelesspoolie: sorry!00:27
lifelesspoolie: if you pull my hydrazine/cron, it will show you the last comment00:27
lifelessso you can see if its been sent to pqm or whatever00:27
lifelessok, first pass reviews done00:28
lifelessI'm going to see if I can finish a spike I started in the weekend to make completing proposed things easier.00:29
lifelesslosa - did something kill stuff on balleny ? I just got an interrupted test run...00:44
lifelessspm: ^?00:51
=== FryGuy_ is now known as FryGuy-
AfCbzr: ERROR: No such file: u'/home/andrew/vcs/java-gnome/.bzr/repository/obsolete_packs/': [Errno 2] No such file or directory: '/home/andrew/vcs/java-gnome/.bzr/repository/obsolete_packs/'03:26
AfCis that normal?03:26
lifelessyou've deleted that directory.03:37
lifelessDon't do that.03:37
lifelessand you should recreate it.03:37
lifelessjam: thnaks04:23
AfClifeless: maybe bzr could create the directory if it's missing?04:28
lifelessAfC: we felt it was better not to04:28
AfClifeless: hm04:28
lifelessdirectories going missing is a pretty nasty event for the filesystem to do to us04:28
AfClifeless: and that error saying the same thing twice is better?04:29
AfClifeless: yes, I can understand that04:29
AfCalthough we've been telling people to "delete the obscelete_packs directory" and if you're saying the nuance is to delete it's *contents* perhaps it's not uncommon to lose the directory?04:29
GaryvdMMorning all04:33
* GaryvdM having a sleep fail :-(04:34
lifeless:<04:41
GaryvdMReading bug reports - sleep fail over :-)04:42
lifelessAfC: we've not been telling people to do that. Some folk have said it, I certainly never havwe.04:43
AfClifeless: oh04:43
AfCI must have misread what I found in the list archive then04:44
lifelessthere are people that run pack, for no particularly good reason04:45
lifelessexcept that you had to in git to get performance, at one point04:45
lifelessand pack keeps the old files around because some fs's (NFS, ext4) are shonky04:45
AfCit must have been from when I had a 5GB .bzr directory for a 20 MB project.04:45
lifelessif people want to *immediately* gc, they can delete the contents of that dir04:46
lifelessbut bzr will gc it after a few commits anyway04:46
lifeless(5 on average)04:46
* igc dinner09:56
=== joerg is now known as Guest70592
kostja_osipovhello. How do I find out the current format of my bzr repo?10:24
kostja_osipovI seem to be unable to find anything in the docs...10:24
spiv"bzr info -v"10:25
kostja_osipovthanks!10:25
bialixheya10:28
=== bialix_ is now known as bialix
quicksilverbzr doesn't deal very elegantly with the case where a directory is removed in one branch and some of the files inside are modified in the other12:14
quicksilverit deals *correctly* just not very elegantly12:15
quicksilverthe messages you get don't immediately tell you what's happened.12:15
lifelessplease file a bug saying that12:16
lifelessgnight12:16
=== oubiwann is now known as oubiwann_
=== oubiwann_ is now known as oubiwann
=== JFo-swap is now known as JFo
=== Guest70592 is now known as joerg
=== gnomefreak76 is now known as gnomefreak
=== deryck is now known as deryck[lunch]
=== oubiwann is now known as oubiwann_
=== oubiwann_ is now known as oubiwann
=== deryck[lunch] is now known as deryck
Spyzer due to interruptions in my internet connection my downloading from bazaar "bzr branch lp:inkscape" is interrupted. How do i resume the downloading from the interrupted point???19:37
Spyzerplease help19:37
Spyzer??19:42
Spyzeris there no possible way of what i ask??19:43
sebi`hi, is there a command to blacklist a file (read: preventing a file to appear in the branch)?19:44
sebi`since I have a bunch of gen*.sh files, which are mostly useless to the enduser19:45
=== radoe_ is now known as radoe
Spyzerhow to ab\void downloading history in a bzr branch ??20:06
Spyzer*ab\void avoid20:06
hnoSpyzer: currently only by having the history locally already...20:10
hnobeen discussions about supporting shallow branches / history horizons, but not sure what the current status of that is.20:11
Spyzerhno: if i do a bzr branch lp:inkscape and it interrupts in middle and if later on i use the option --use-existing-dir later on. Shall that resume the donwload the branch form the point it was terminated20:11
Spyzer?20:11
Spyzerpls don't mind the typos20:12
Spyzerits saying bzr: ERROR: Already a branch: "inkscape"20:13
hnoSpyzer: No idea, but I guess not.20:13
Spyzerhow do is\relove this20:13
Spyzeroh man, is there no way to resume a download20:13
Spyzer?20:13
hnoIt's only initial branch creation which is this heavy. If you already have branched some branch of the same project before then no need to download much again.20:17
sebi`anyone? ):20:19
Kohlrabisebi`: bzr ignore?20:20
sebi`Right, thanks.20:20
mkanatspm: Did codebrowse get updated to trunk after I mentioned I fixed the fix better, the other day? (A few weeks ago.)20:47
lifelessmoin20:56
lifelessjam: would like to talk network fetch efficiency if you're around20:57
* hno wonders if bzr wouldn't benefit from a general cache...20:58
GaryvdMvila: ping21:00
TresEquishno:  you have a cache lying around ;)?21:03
jamlifeless: morning. I'm trying to get reviews done, but otherwise I'd appreciate the discussion21:03
lifelessjam: code or peers ?21:04
lifelesspeers we have until june 11 I think21:04
jampeers21:05
lifelessso tell me when you have a break21:06
jamlifeless: did you make the peer requests, or was it done for you?21:06
lifelessyou make them yourself - on the left bottom there is 'add peer'21:06
lifelessor something to that effect21:06
lifelesswe had to have the mgr review sent in monday, then there is a gap21:07
hnoTresEquis: Non-obvious how to make a cache to bzr with all the smartcalls around..21:08
jamlifeless: so what's up?21:36
lifelesswell21:36
lifelesswe seem to be sending 2-3 times the repo size on smart push of a full branch21:36
lifeless200MB to push bzr.dev to launchpad when its not stacked21:36
lifelessI struggle to believe that its the encoding overhead21:36
lifelessand, possibly related, we're not saturing the network at all effectively21:37
lifelessI'm patch pilot this week, so not all that well placed to just nose down and drill into this.21:37
lifelesss/uring/urating/21:37
lifelessI'm hoping that you or spiv will be between patches and able to look deeply into it21:38
mkanatlifeless: I think a smart pull of a full branch is the same, as well.21:39
jamlifeless: I'm 75% confident that it is the "split to organize 'properly'" code, and dependent on how the sort order turns up. If you want a quick and dirty test, just change the fetch order to 'unordered' and see if it doesn't drop the bytes transferred.21:39
lifelessmkanat: yes, I'd expect that21:39
mkanatlifeless: Oh, yeah, I suppose because of the indexes and all.21:39
jammkanat: no indices for smart pull, but push should transfer ~ the same as pull21:40
lifelessmkanat: just because push and pull on the smart server are symmetrical, its just which end does the work21:40
lifelessjam: so I can do quick tests21:41
lifelessjam: what I want, is to know that someone else is going to run it down to ground ;)21:41
jamlifeless: I've looked at this a few times, and mentioned it, and didn't get much traction anywhere...21:42
lifelessjam: traction with people or code ?21:42
jampeople21:42
jamIf we really want minimal transfer then we need to compress on the server isde21:42
jamside21:42
lifelessok21:42
jamIMO21:42
lifelessmakes sense to me21:42
lifelessI've been saying lp needs to scale with multiple front-end CPU boxes for a while, so it won't be a total suprise there ;)21:43
jamI think we were avoiding it because of wanting to avoid load on a central location21:43
jamthough still a question as to whether push or pull is more common :)21:43
jammost likely fetching from lp is the most common.21:43
lifelessI don't recall any decision to avoid it21:43
jamlifeless: we've certainly discussed wanting clients to do a bit more work than a server, since you usually have N clients and 1 server.21:44
lifelessthats true21:44
lifelessbut, if wishes were horses ;)21:44
lifelessare there any stats we should be gathering from the server to tell if this is the case ?21:45
jamlifeless: the "split group" comments would be a likely point21:45
lifelessok21:45
jamsince it is taking existing groups, and splitting them into less efficient ones21:45
mkanatjam: If there are no indices for smart pull, then what makes the pull so large?21:45
jamthe big thing that I've seen21:45
lifelesswell something I can follow through is getting some better logging and stats21:45
lifelessmkanat: bugs ! :P21:45
mkanatlifeless: Hahaha, okay. :-)21:46
jamis when it cycles between 2 groups over and over21:46
jamtaking some bits from A, then B, then A, then B21:46
jamthose are definitely going to put a lot more data on the wire, as each split == fulltext21:46
lifelessso, CHKRepo format is 'unordered' already21:47
lifelessand StreamSource is groupcompress21:47
jambzrlib/repo_fmt/groupcompress_repo.py line 102521:48
jamset that to 'unordered'21:48
lifelessyeah21:48
lifelessvila: thanks - Working tree "/home/robertc/source/baz/bzr-test-fixes/" has uncommitted changes (See bzr status). Uncommitted changes will not be pushed. - much better.21:49
lifelessok, finding revisions taking a bit21:51
jamlifeless: so I just pushed on the local network, and saw 101.5MB to transfer a random bzr branch.21:52
jamsize on disk when finished was 36MB21:52
lifeless3x21:52
jamwell, size in 'upload'21:52
jamno indices written yet21:52
lifelessyes21:52
jamstill spinning on the 'commit_write_group' I think21:52
lifelessyes, we need to address that too21:52
jamlifeless: one other thing to 'address' is that I think the 'should this be rebuilt' heuristic is wrong for chk pages. As it checks to see if a group is 'full enough' and if not, it rebuilds it. But chk groups got better compression by logical grouping than trying to make the groups max size21:54
lifelessok21:54
jamhmm... I just waited maybe 5-10 min after pushing, and now it is on Missing Keys. Is it going to be doing the check *yet again* ...21:55
lifelesswould that affect the A<->B<->A behaviour you mentin abve ?21:55
jamlifeless: I don't think so21:55
jamI'm guessing the A B A stuff is more because of inexact sorting21:55
jamThe sort is stable if you have the same set of revisions21:55
lifelessok, streaming @ 70KB21:55
jambut I don't know how stable it is when new stuff is added.21:56
lifelessI'm on adsl, so thats maybe 1/3 of my available b/w21:56
jamlifeless: well, I'm testing on the home network, and certainly not saturated.22:00
jamthough I'm in the basement with the WAP 2 stories up22:00
jamlifeless: sending from a freshly populated repo, which only has the content in question22:02
jamand I see 52.9MB transferred22:02
jamnote that size on disk post indicies is 50MB22:02
jam12M in indices, 39M in packs22:02
jamlifeless: in the latter case, I see 18: "creating new compressed block" lines in .bzr.log, in the former, it is closer to 3,30022:05
jamlifeless: case 3, push from unorganized repo, using 'unordered' texts, shows about 73.2MB transferred22:12
lifelessI see many pages22:14
lifelessnot 3.3K worth22:14
lifeless12 pages22:14
lifeless25 per page - 30022:14
lifeless1082.823  Transferred: 68957kB (63.8kB/s r:2368kB w:66588kB)22:15
lifelessthat was with unordered22:16
lifelessso I image it would be considerably worse without22:16
lifelessI can sftp upload a file at roughly the same speed22:17
lifeless+- < 10%22:17
GaryvdMvila: I've merge lp:~garyvdm/qbzr/annotate_find into lp:qbzr22:36
jamlifeless: @64kB?22:38
jamlifeless: so your unordered and my unordered seem on par22:38
guijemontGaryvdM: hi, i have a qlog question22:38
lifelessyeah, 68, 71, etc22:38
GaryvdMHi guijemont.22:38
lifelessjam: no, unordered is ~= SFTP22:38
lifelessusing lftp to upload a single pack22:39
jamlifeless: I mean that uploading via lftp you get 64kB/s22:39
guijemontGaryvdM: when expanding a node, some other nodes (with links to the expanded stuff) are expanded22:39
lifelessso, I'm getting wire speed with unordered22:39
lifelessor at least, wire + latency to london22:39
jam... Transferred ... (63.8kB/s ...)22:39
guijemontbut there seems to be a limit on the quantity of branches that is expanded22:40
guijemontbut I don't get what that limit is nor how/where it is implemented22:40
lifeless`../.bzr/repository/packs/2fc2a401beb2fd09d041842e3c27ebb7.pack' at 101785571 (76%) 61.4K/s eta:7m [Sending data]22:40
GaryvdMguijemont: ah - let me have a look at the code.22:40
guijemontwhat I was looking at is colapse_expand_rev(), but I'm not sure it's the right place22:41
GaryvdMguijemont: That is the right place22:41
guijemontesp. since it seems ot be called with visible=True, which results to  a quite simple codepath22:41
GaryvdMguijemont: It starts by looking at rev.twisty_branch_ids22:46
GaryvdMguijemont: rev.twisty_branch_ids gets updated each time compute_graph_lines runs22:47
guijemonthmm, and does that explain how some parents get expanded, and some other not (apparently, with lines drawn with dots)22:52
guijemont?22:52
GaryvdMguijemont: It remebers what caused a branch to expand.22:54
GaryvdMguijemont: e.g.: http://pastebin.org/27615522:54
guijemontoh, ok, and it doesn't go more than a given number of recursions into that?22:55
GaryvdMguijemont: if you expand 4, and then 3 and then collapse 3, branch line 1.1 will stay open, because 4 expanded it22:56
guijemontyeah, i noticed that22:56
GaryvdMguijemont: but if you expand 3 and then 2.1.1, then collapse 3, branch line 1.1 will collapse22:57
GaryvdMguijemont: This is what self.branch_lines[branch_id].expanded_by is for22:57
GaryvdMguijemont: If collapsing, the we 'recurse' into branch_line.merges22:59
guijemontok22:59
guijemontnot exactly my issue though22:59
guijemonthold on22:59
guijemontmaking a screenshot23:00
GaryvdMguijemont: processed_branch_ids is to track what we have looked at so that we don't do a branch more than once, which prevents a infinite 'recursion'/loop.23:01
guijemontGaryvdM: http://emont.org/dots.png23:02
guijemontthere, some lines are drawn with dots23:02
guijemontand some of them go to the "merge level 0 ancestor" of the bzr parent23:02
guijemontwithout expanding happening23:02
GaryvdMguijemont: hmm...23:03
GaryvdMguijemont: the dots indicate that the line is going to a non intimidate ancestor.23:04
GaryvdMguijemont: what happens if you click on say 2972.4.2, and then click on the other parent in the lower plane?23:05
guijemonthold on, i closed the window...23:06
guijemontand what's a "non intimidate ancestor"?23:07
GaryvdMguijemont: The line layout algorithm I use dose not suit the mysql dags well. :-(23:07
guijemontwell, I guess mysql is a good test23:08
guijemontsince I have that bug with it with my change to viz23:08
guijemontbecause my method to expand to the parents23:09
guijemontis to do it one level at a time23:09
guijemontin a callback that gets called when a row is expanded23:09
guijemontso, for each new row that I expand, the callback gets called again23:10
guijemontbut in the case of mysql, the depdth at which you go with that is too big23:10
guijemontand I end up with a maximum recursion exception23:10
guijemontso i wanna try to do that iteratively instead23:11
GaryvdMguijemont: e.g.: http://pastebin.org/276155 - if branch line (bl) 1.1 is visible, and bl 2.1 was hidden, we would show a dotted line from 3 to 1.1.1. 1.1.1 is not a parent of 3, but rather a distant ancestor...23:11
GaryvdM=  "non intimidate ancestor"23:11
guijemontGaryvdM: do you know of a document that explains that kind of vocabulary?23:12
fullermdIt's not polite to intimidate your ancestors.23:15
GaryvdMguijemont: no sorry, these are things that I've kind of made up. I'm sure there are definition of these things in formal graph theory, which I have unfortunately not studied.23:15
guijemontok23:16
fullermdITYM "immediate"...23:16
GaryvdMfullermd: yes - thanks23:16
guijemontahh23:17
guijemontoh23:17
guijemontok23:17
guijemontmight be clearer with "immediate"23:17
GaryvdMjam is seems very knowledgeable about graph theory. Maybe he can give us the correct terms.23:17
* fullermd thinks there need to be more terms like "weird uncle".23:18
GaryvdMguijemont: In the code, I refer to non immediate ancestor as direct = False23:19
fullermdHow about 'indirect'.23:20
GaryvdMdirect = False / indirect = True23:20
fullermdI mean as a term.23:20
guijemontI'm sure I *knew* all this graph theory vocabulary well23:21
fullermdMakes sense as the opposite of direct.23:21
guijemontthough I knew it in french...23:21
GaryvdMguijemont: It also does not help that I'm terrible at spelling :-(23:22
guijemonthu23:23
guijemontbtw, I think colapse_expand_rev() should be called collapse_expand_rev() ;)23:24
lifelessyes23:24
GaryvdMyes - that is one of my spelling mistakes....23:24
guijemontanyway, I think I get how the dotted lines work now23:24
guijemontat least the idea, which is what interests me now23:25
guijemontGaryvdM: thanks!23:25
GaryvdMguijemont: Pleasure.23:25
GaryvdMSpelling of 'collapse' fixed. :-)23:34
=== sebi_` is now known as sebi`
guijemontcool, so I did something useful today, i reported a bug that got fixed :)23:40
guijemontnow I feel productive23:41
guijemontI can go to bed23:41
lifelessmwhudson: hi23:43
mwhudsonlifeless: hello23:43
lifelessmwhudson: I'd like to get stats on bzr smart server stuff on production23:43
lifelessI'm sure you've considered this.23:43
lifelesscare to tell me the shortest path ? :)23:43
mwhudsonlifeless: 'stats' ?23:47
lifelesshow many split groups per push/pull23:47
lifelessvs unsplit groups23:47
lifelesstotal bytes sents23:47
lifelesspossibly drilling in further23:48
mwhudsonlifeless: i guess something like:23:48
* mwhudson thinks a bit more23:48
lifelesslike23:48
lifelessI'm not scared if you say 'change bzr, change bzr-lp-serve, change oops, and change this odd crnoscript23:48
mwhudsonlifeless: i guess something like: arrange for the bzr serve processes to log somewhere more obvious than ~/.bzr.log23:48
mwhudsonlifeless: then make sure that the bzr serve processes log the information you want to see (possibly with a -D style flag i guess?  maybe this already exists)23:49
mwhudsonlifeless: arrange with sysadmins to copy log files somewhere you can see them23:49
mwhudsonlifeless: write scripts to parse said log files23:50
mwhudsonlifeless: none of these steps seems particularly hard23:50
lifelessok23:50
lifelesswhats unobvious about ~/.bzr.log ?23:50
mwhudsoni got a bit frustrated at one point because i wanted to have the bzr serve process log to a file like bzr-serve-$PID.log but by the time you know the pid it's too late to set $BZR_LOG_FILE (or whatever that's called)23:51
mwhudsonlifeless: well, perhaps 'obvious' was the wrong word23:51
lifelessok23:51
lifelesswhat would a better word be?23:51
mwhudsonlifeless: it has the output of all the bzr serve processes jumbled up in it, it's not synced anywhere, and it gets rotated by bzr's policy not the sysadmin's23:52
lifelessok, the first point is fine (we put pids in), the sync issue we can fix, the rotation however - we should fix that in bzr.23:52
lifelesslog_rotation = None in ~/.bazaar.conf ?23:53
lifeless.bazaar/bazaar.conf, I mean23:53
mwhudsoni guess that works23:53
lifelessWe can work up to other things23:53
lifelessalternatively23:59
lifelessperhaps23:59
lifelesslog_file = ~/.bazaar/logs/$time-$pid.log23:59
lifelessin bazaar.conf23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!