| === jtv1 is now known as jtv | ||
| === allenap` is now known as allenap | ||
| * Daviey ponders mongo... | 08:50 | |
| Daviey | you know.. Ubuntu isn't ready for mongo do be a dependency.. | 08:51 |
|---|---|---|
| Daviey | bigjools: ^ | 08:51 |
| bigjools | Daviey: fancy that | 08:53 |
| bigjools | jtv: how may I help? | 08:53 |
| Daviey | bigjools: I assume current quantal version is the release that has been developed against ? | 08:57 |
| Daviey | (another project asked us to consider a major version update) | 08:57 |
| bigjools | Daviey: yeah - is someone pulling in mongo? | 08:57 |
| Daviey | bigjools: Are we not? | 08:59 |
| bigjools | Daviey: AFAIK, no | 09:00 |
| bigjools | not right now anyway | 09:00 |
| Daviey | we will, no? | 09:00 |
| bigjools | Daviey: possibly - I don't recall the exact discussions that jam1 had around this | 09:01 |
| mgz | Daviey: we decided to avoid the excitement | 09:01 |
| mgz | at least until 13.04 | 09:02 |
| Daviey | mgz: What what ding ding? | 09:02 |
| mgz | why exactly isn't ubuntu ready for a committed relationship with mongo though? | 09:02 |
| mgz | and it's us that needs to change, not them? | 09:02 |
| Daviey | mgz: you make it sound like a marriage :) | 09:02 |
| Daviey | I think we should see other people.. | 09:03 |
| Daviey | mgz: The process for mongo to be in main, hasn't been started... that is all. | 09:03 |
| mgz | there are plenty of other databases in the sea | 09:03 |
| lifeless | hah | 09:03 |
| lifeless | not sure that that analogy holds | 09:03 |
| === jam1 is now known as jam | ||
| rbasak | Do we have a preferred method of defining properties? Is @property def foo, then @foo.setter OK? | 09:14 |
| mgz | jam: https://codereview.appspot.com/6569053 | 09:18 |
| bigjools | rbasak: I think that's ok | 09:19 |
| jam | mgz: hmm.. I don't see a way to sub to just the MP without commenting on it. | 09:19 |
| jam | mgz: is it possible to call it just 'tags'? rather than 'maas-tags' ? | 09:20 |
| jam | I believe we want to match the search in our web UI with what you pass to juju, and putting 'maas-' in there is a bit annoying. | 09:20 |
| jam | (I recognize the idea of namespacing, though) | 09:20 |
| jam | mgz: also here: https://codereview.appspot.com/6569053/patch/1/5 you seem to be removing 'maas-name' but it is still listed as supported in the other files | 09:22 |
| jam | or is it just that you fallback to None if not supplied? | 09:22 |
| jam | mgz: and here: https://codereview.appspot.com/6569053/patch/1/6 I think we are going to go with 'tag1,tag2,tag3' style rather than tag1&!tag2|tag3 | 09:23 |
| mgz | jam: there's a linked lp proposal you can comment on if their funny code review doesn't play ball | 09:23 |
| mgz | https://code.launchpad.net/~gz/juju/add_maas_constraints/+merge/126203 | 09:23 |
| jam | the changes look good to me, just some small bits | 09:24 |
| mgz | I think in the context of the maas code, just using "tags" is fine, but it's so generic a term I don't like exposing it as a general constraint without getting more specific | 09:24 |
| mgz | the removal of maas-name from those tests is just cleanup, it's not required and confuses what they're trying to test | 09:25 |
| mgz | my first pass at this parsed and validated the tags on the juju side | 09:26 |
| mgz | but I reverted to this approach of just passing a string through, because I'm not sure how far we want to go with tag combinations | 09:26 |
| jam | mgz: well I know hazmat asked us to make it less specific, so the provider should query Maas for a list of acceptable tags. | 09:27 |
| mgz | could just pick nodes with a superset of the tags specified, or could do full boolean logic | 09:27 |
| jam | less generic? | 09:27 |
| mgz | I think hazmat will be fine with maas returning good errors for non-existant tags or a bad tag string | 09:27 |
| jam | The argument is that deploying is async, so by the time the provider responds, it is too late to inform the user of a typo/etc. | 09:27 |
| mgz | right, so provided we catch before acquire actually happens that a tag is bad, and return a 4XX straight away without going down to the cluster, I think that's fine | 09:28 |
| mgz | ...I probably want to add some tests for that error case | 09:30 |
| jam | mgz: shall we chat on mumble/ | 09:38 |
| jam | ? | 09:38 |
| bigjools | Daviey: should I reference the snakeoil certs as an example when I stick in SSL support? (so it just works as a demo) | 09:41 |
| mgz | jam: lets | 09:45 |
| Daviey | bigjools: i'd say so | 09:46 |
| Daviey | hmm | 09:46 |
| Daviey | what do you think? | 09:46 |
| bigjools | Daviey: WFM | 09:46 |
| bigjools | it works, anyway :) | 09:47 |
| jam | mgz: http://ezix.org/project/wiki/HardwareLiSter#Howtointerpretlshwsoutput | 10:02 |
| rbasak | rvba: having some trouble migrating Node.architecture into Node._arch with a Node.architecture frontend property. It works, except NodeForm screws up. I think it's because it can't automatically generate the form from the model for the architecture property, since that's no longer an ORM field. | 10:56 |
| rbasak | rvba: can I perhaps make NodeForm.architecture an "extra" field which does map to Node.architecture's property but not automatically? | 10:57 |
| rvba | rbasak: sounds like a good idea. You'll have to create a form field which handles that logic: display a single choice but stores into two fields. | 10:58 |
| rbasak | rvba: to start with I'm just trying to move architecture into _arch but with the same behaviour as before | 10:59 |
| rbasak | So there's an abstraction layer that I can then modify. But all tests should pass at this stage | 10:59 |
| rvba | rbasak: let me have a look at NodeForm. | 11:00 |
| rbasak | rvba: I'm not clear on the exact mechanics between ModelForm, Form and Model | 11:00 |
| rvba | rbasak: it overrides the 'architecture' field so you'll need to rename that field as well. | 11:00 |
| rvba | rbasak: ModelForm is only a base class for forms that help creating forms from model definitions. | 11:01 |
| rvba | That's the 'Meta' part where you define on which model a ModelForm is based and which fields are taken from that model. | 11:02 |
| rbasak | rvba: yep - so if I remove fields = ( 'architecture') from NodeForm, it should still work? I'm a bit confused as to what purpose it's serving at the mment, since there's already an explicit architecture definition in NodeForm | 11:02 |
| rvba | rbasak: the ModelForm also handles the fact that when you save that form, an specific object is needed. | 11:03 |
| rvba | rbasak: s/needed/created/ | 11:04 |
| rbasak | rvba: so if I remove fields=('architecture'), then the object will be created but without that field? How would I override that to add it again? | 11:05 |
| rvba | rbasak: having 'architecture' in fields (even if the form field is defined on the form itself) is to instruct the form that whatever comes out of the form's field 'architecture' is a field on the created/modified model object. | 11:07 |
| rbasak | rvba: thanks. I think I understand that part then, but don't understand the failure I'm getting | 11:08 |
| rbasak | rvba: lots of ValidationError: {u'architecture': [u'This field is required.'] | 11:08 |
| rvba | rbasak: I think you want to create a multivalue field (https://docs.djangoproject.com/en/dev/ref/forms/fields/#multivaluefield). | 11:08 |
| rvba | rbasak: can I see the diff? | 11:08 |
| rbasak | rvba: http://paste.ubuntu.com/1226381/ | 11:09 |
| rvba | rbasak: right, but NodeForm still has a required 'architecture' field. Hence the validation failure. | 11:10 |
| rbasak | rvba: but I still require an 'architecture' to be set in the model | 11:10 |
| rbasak | rvba: ah. Is it that model_to_dict isn't returning an 'architecture' field? | 11:14 |
| rbasak | rvba: one failure is in NodeGroupInterfaceHandler.update for example. Perhaps it's not retrieving the architecture field, so when it tries to update it updates without one | 11:15 |
| rvba | rbasak: yep, something like that. But I'm trying to work out if the problem is in Django itself (which would mean it can't work with python properties) or in how MAAS handles the forms in the API. | 11:16 |
| rbasak | rvba: sqlalchemy has a solution to this it calls hybrid properties, which allow derived attributes to be defined in the model and mapped properly. | 11:17 |
| rbasak | rvba: I found http://stackoverflow.com/questions/12217763/does-django-orm-have-an-equivalent-to-sqlalchemys-hybrid-attribute which suggests that django doesn't have an equivalent | 11:17 |
| rvba | rbasak: also, without the db migration in place, South hijacks the models definition so your new field is probably not even properly registered. | 11:24 |
| rbasak | rvba: I don't have a new field! | 11:25 |
| rbasak | (yet) | 11:25 |
| rvba | Right, but I'm wondering if the change is still something that South needs to know about. | 11:25 |
| rbasak | ok | 11:25 |
| allenap | mgz: You can self-review https://code.launchpad.net/~gz/maas/move_update_hardware_details_tests/+merge/126226 if you want; there's nothing controversial in there. | 11:25 |
| rbasak | rvba: looks like this might be an alternate approach to using properties: Challenger 600 | 11:30 |
| rbasak | Aarh | 11:30 |
| rbasak | https://docs.djangoproject.com/en/dev/howto/custom-model-fields/ | 11:31 |
| rvba | rbasak: did you consider keeping the architecture field as is but have it populated with things like 'arch/subarch' and have two methods on the node to get just the arch or just the subarch? | 11:31 |
| rbasak | rvba: I thought it would be cleaner if everything apart from the model dealt with the architecture as a 2-tuple | 11:31 |
| rbasak | rvba: rather than adding another subarchitecture field | 11:31 |
| rbasak | (and needing to pass two values around all the time) | 11:32 |
| rbasak | But perhaps this is more complicated than it first seemed | 11:32 |
| rvba | But what about storing both the arch and the subarch in the 'architecture' field? | 11:32 |
| rbasak | Oh | 11:32 |
| rbasak | Using a '/' as a separator? | 11:32 |
| rvba | Yes | 11:32 |
| rvba | 'arch/subarch' | 11:33 |
| rbasak | That would work too but I think even more ugly. At what layer would we translate to a 2-tuple? And if we don't translate to a 2-tuple, then code that needs to just match arch will have to split('/') and we'd end up with those all over the place | 11:33 |
| rvba | What part of the code needs a 2-tuple? | 11:34 |
| rvba | Probably only a fraction of the code. | 11:35 |
| rvba | All the rest of the code (UI/API/model) could deal with 'arch/subarch'. | 11:35 |
| rbasak | Any time I want to say "if arm", I'd need to say "if architecture.split('/')[0] == 'armhf'" | 11:36 |
| rvba | If the code that needs to do that is manipulating node objects, then this could be a method of Node. | 11:40 |
| rbasak | I suspect that the node object itself will be lost higher up in the call stack, and only the architecture field will be passed down | 11:43 |
| rvba | Yeah, probably. | 11:43 |
| rbasak | But it's a trade-off, and implementing a 2-tuple may be too complicated | 11:43 |
| rbasak | The only way clean way right now seems to be a django custom field | 11:43 |
| rbasak | If overloading the string type is acceptable to the team, I'll happily do it that way | 11:44 |
| rbasak | allenap: ^^ your opinion please? | 11:44 |
| * allenap looks | 11:45 | |
| rvba | rbasak: the problem you're having right now is that the ModelForm only tries to save on the node object the fields that a) are listed in 'field' and b) correspond to a real DB field. | 11:46 |
| rvba | s/'field'/'fields'/ | 11:47 |
| rbasak | rvba: I think I follow (just) | 11:47 |
| rbasak | rvba: so it seems that using a property on a Model in order to map derived fields is not so straightforward in django | 11:47 |
| rvba | Indeed. | 11:47 |
| rbasak | rvba: I'm concerned that there will be other areas where doing it this way might be a problem in future even if we do manage to make it work | 11:48 |
| rvba | And a custom field is only really helpful if you're trying to store a structure inside a single db column. | 11:48 |
| rbasak | ah | 11:48 |
| rbasak | well that eliminates that option then | 11:49 |
| rbasak | so it looks like overloading a string is the only option left | 11:49 |
| rbasak | I'll just do that then | 11:50 |
| rbasak | At least it's easier even if it is ugly! | 11:50 |
| rbasak | Unless allenap has a better suggestion? | 11:50 |
| rvba | I think this will be much more simple as a single field. | 11:51 |
| allenap | rbasak: I was thinking of something like http://paste.ubuntu.com/1226429/ | 11:51 |
| rvba | Even if it's slightly ugly. | 11:51 |
| rvba | allenap: that's the cleanest possible in terms of db but it will make the form code complicated. | 11:52 |
| allenap | I didn't know how far back to read, so you might have discounted that idea already. | 11:52 |
| rvba | Because now you've got one field which is dependent on the other. | 11:52 |
| rbasak | allenap: yeah that's exactly what I tried, but it breaks model_to_dict | 11:52 |
| allenap | rvba: Does the form code have to be driven from the model definition, or can its "architecture" field be specially prepared? | 11:53 |
| rbasak | allenap: and it feels that trying to fix that up is a step too far, and will end up being more hacky than just overloading the string | 11:53 |
| allenap | Yeah. | 11:53 |
| rvba | allenap: it can be done at the form level. | 11:54 |
| allenap | Just put "arch/subarch" in the field, and have a couple of functions to get the arch and subarch out. I guess that was the intention behind subclassing str/unicode. Can Django be taught to always use a particular string class for a field? | 11:54 |
| rvba | allenap: "subclassing str/unicode"? | 11:55 |
| rbasak | oooh | 11:55 |
| allenap | rvba: I thought that was what rbasak had suggested earlier with "overloading the string type". | 11:56 |
| rbasak | that's not what I meant, but it's an interesting idea | 11:56 |
| rbasak | or just have an Architecture class which can be contructed with a string | 11:56 |
| allenap | It would be nice to avoid impedance problems when dealing with Django's models, if we can. | 11:57 |
| allenap | Custom field type territory then. | 11:58 |
| rbasak | yeah | 11:58 |
| rbasak | but it would work | 11:58 |
| rbasak | Is it worth the effort? | 11:58 |
| rbasak | Or can I just use a string for now? | 11:58 |
| allenap | Again, not sure if the form machinery will cope. | 11:58 |
| rvba | The form machinery should be fine I think. | 11:59 |
| allenap | rbasak: Just use a string I think :) | 11:59 |
| rbasak | Although a custom field would save a db migration | 11:59 |
| rbasak | though maybe we should do one anyway | 11:59 |
| rvba | A db migration is really cheap. | 11:59 |
| rbasak | I presume we can do a migration even without a db schema change? | 12:00 |
| rbasak | update node set architecture=architecture + 'generic' | 12:00 |
| rbasak | update node set architecture=architecture + '/generic' | 12:00 |
| rvba | You want a datamigration (and not a schema migration). | 12:00 |
| allenap | I have to go and feed children, back later. | 12:00 |
| rbasak | ok | 12:00 |
| rbasak | I think that's a decision then | 12:01 |
| rbasak | thank to both of you! | 12:01 |
| rbasak | thanks | 12:01 |
| mgz | this is an error? | 12:06 |
| mgz | While: Installing repl. Getting distribution for 'amqplib>=0.6.1'. | 12:06 |
| mgz | Error: Picked: amqplib = 1.0.2 | 12:06 |
| mgz | what is buildout trying to tell me? | 12:08 |
| jtv | mgz: it's configured not to pick its versions freely, yet the versions we specify are minima, not exact values. We fixed it by relying on installed packages more. | 12:21 |
| jtv | Removing the egg from your cache may help. | 12:22 |
| mgz | that's not an egg, it's what precise installed I suspect | 12:24 |
| jtv | oh, precise | 12:24 |
| mgz | but... it really can't do > properly? really? | 12:24 |
| jtv | It's configured not to. | 12:24 |
| jtv | Don't ask me for the details; I don't know them. | 12:25 |
| jam | jtv: I think your patch is breaking Jenkins. Did you see the failures? | 12:25 |
| jtv | jam: not yet -- I'll have a look. | 12:25 |
| jam | https://jenkins.qa.ubuntu.com/job/maas-trunk/849/console d | 12:25 |
| jam | has: u'refresh_workers' not in '--======== | 12:25 |
| jam | in a 'test_start_up_calls_refresh_secrets" | 12:25 |
| jam | jtv: for what it is worth, that test fails locally for me as well. | 12:25 |
| jam | It looks like it is a mix of your test plus allenap's change to use email | 12:26 |
| jam | (given that it is asserting a string is present in a mime encoded chunk) | 12:26 |
| jtv | It sounds as if registration of tasks got messed up somehow | 12:28 |
| mgz | removing the egg alas did not help, precise has 1.0.0+ds-1, buildout gets and installs 1.0.2 then complains that 1.0.2>=0.6.1 is false... | 12:29 |
| jtv | No, trunk is for quantal now. | 12:30 |
| jtv | jam: ahhh, looks like something in the data presentation changed -- may have been Gavin's work on maascli. | 12:33 |
| jam | jtv: or his recent change to use the email module? | 12:34 |
| jam | I wouldn't usually expect to do 'content in mime_string' | 12:34 |
| jtv | dunno about that | 12:34 |
| jam | (given that mime_string might be base64 encoded) | 12:34 |
| jtv | Yeah -- and mime is such a pain to deal with | 12:34 |
| jtv | :( | 12:35 |
| jam | jtv: since you're here, I'm trying to add a test that if you give bad data, we make sure the db stays consistent. | 12:35 |
| jam | I am able to trigger the DatabaseError | 12:35 |
| jam | but now I don't know how to rollback | 12:35 |
| jtv | otp though | 12:35 |
| jam | in order to inspect the db to make sure the content is consistent | 12:35 |
| jam | rvba: ^^ | 12:35 |
| jam | I see there is 'transaction' objects in Django, do I just need to use it in the test suite? | 12:35 |
| rvba | jam: you need to use a special test case if you want to use transaction management in your test. | 12:36 |
| jam | rvba: well, there are 2 levels then. I also want to test that the API call doesn't bork things up in the middle, which means it might need a transaction object as well. | 12:37 |
| rvba | This is because the default testcase in Django uses rollback to wipe out the data between tests. More details here: https://docs.djangoproject.com/en/dev/topics/testing/#django.test.TransactionTestCase | 12:37 |
| jam | rvba: ah, so none of the 'obj.save()' calls are actually being committed? | 12:38 |
| jtv | that's not a commit | 12:38 |
| jtv | that's just "write this to the database" | 12:38 |
| jtv | doesn't care one hoot about transactions. | 12:38 |
| jam | jtv: https://docs.djangoproject.com/en/dev/topics/db/transactions/?from=olddocs (by default .save is committed in essentially autocommit mode) | 12:38 |
| jtv | The same goes for the "commit" argument to form.save(): it's about writing stuff to the db, not about transactions | 12:38 |
| jtv | Not "essentially" autocommit, but actual autocommit. | 12:39 |
| jtv | It's entirely divorced from transaction management. | 12:40 |
| jam | jtv: right, I was trying to ascertain what is being done in Maas. | 12:40 |
| jtv | OK | 12:40 |
| jam | I do understand the difference between a .save() and a .commit. | 12:40 |
| rvba | jtv: right, unless you're using the TransactionTestCase. | 12:41 |
| jtv | (Sorry, I'm in the next phone call, a friend who seems upset but there doesn't seem to be anything to do for him but let him tallk. Very distracting) | 12:41 |
| jam | for the API calls, what is the transaction logic? Autocommit? The "TransactionMiddleware" that is recommended. | 12:41 |
| rvba | err, jam ^ | 12:41 |
| allenap | jtv: Want me to fix that failing test? I know what to do, and have a function ready to do it. | 12:41 |
| jtv | Oh, that'd be great! | 12:41 |
| rvba | allenap: since you're there, mgz seems to be having the same problem I had yesterday. | 12:41 |
| allenap | rvba: The amqp one? | 12:42 |
| rvba | Since then I upgraded to Quantal and the problem is gone. | 12:42 |
| rvba | allenap: yes | 12:42 |
| allenap | Damnable buildout. | 12:42 |
| allenap | mgz: Are you able to upgrade to Quantal, or at least use it on Canonistack? | 12:43 |
| jam | rvba: so is it as simple as importing maastesting.djangotestcase.TransactionTestCase? And not worry about the other setup code in maasserver.testing.testcase.TestCase.setUp() ? | 12:43 |
| jam | allenap: mgz is off to lunch. But he did mention it to me. | 12:43 |
| jam | I think it is a matter of: "why is running on P so horribly broken" vs | 12:44 |
| jam | I cannot run on Q | 12:44 |
| allenap | jam: That sounds like he's not blocked, phew :) | 12:44 |
| jam | I think it is a bit of 'I don't *want* to run on Q as well' but he mentioned being able to switch VMs. | 12:45 |
| jtv | jam: it should be as easy as changing TestCase classes, yes. Just slower. | 12:45 |
| rbasak | rvba: so I'm think of this: http://paste.ubuntu.com/1226522/ - would this work for the web UI? | 12:46 |
| jtv | However we've been seeing some mysterious test failures with unusual invocations of the test suite, and those _seem_ to be related to TransactionTestCase. So whatever you do, run the full test suite! | 12:46 |
| rvba | jam: Good point, you might need to create a class which inherits from the TransactionalTestCase and have the setup from maasserver.testing.testcase.TestCase. | 12:46 |
| rvba | rbasak: that should be ok. | 12:47 |
| rbasak | thanks - I'll continue | 12:47 |
| jtv | jam: about that pastebin -- I'm not sure the migration runs in a transaction, so breakage in mid-transaction might leave you with only some of the nodes updated. | 12:48 |
| jtv | Oh, that's rbasak! | 12:49 |
| jtv | rbasak: don't know how much it matters but those node updates in your schema patch may run in autocommit. | 12:49 |
| rbasak | that's insane | 12:50 |
| rbasak | whose idea was it to make data migrations _not_ run in a transaction by default? | 12:50 |
| jtv | I'm not _sure_ they don't run in transactions. | 12:50 |
| rbasak | why would one ever not want to do that?! | 12:50 |
| jtv | The django people do not seem to have much truck with the whole notion. | 12:50 |
| jtv | I hope my impression is just an exaggeration, but I hear the same from others. | 12:51 |
| rbasak | jtv: we're good: http://south.readthedocs.org/en/latest/migrationstructure.html#transactions | 12:51 |
| jtv | Phew. Thanks, that's a load off my mind! | 12:53 |
| jtv | South != django :) | 12:53 |
| jtv | There's another reason to be pleasantly surprised: don't know what it's like now, but transactionality of schema changes used to be a laughable concept in many databases. | 12:55 |
| jtv | allenap: out of interest, do you know which change broke my test? And what is that fix you had lying around? | 12:56 |
| allenap | jtv: I changed the multipart mime stuff to use the built-in email stuff for generation, but owing to a "bug" in it, I had to ensure that everything is now base64 encoded so that I can post-process line-endings. | 12:59 |
| allenap | jtv: The reason for the change was unicode support. | 12:59 |
| allenap | jtv: The function that's lying around is parse_headers_and_body_with_django. | 12:59 |
| jtv | Ah. That means you understand more of MIME in python than most now. Please while you're at it give us a better replacement for the standard library's MIME bits. :) | 12:59 |
| jtv | Ahhh nice. | 12:59 |
| allenap | However, that's a pita because it requires DJANGO_SETTINGS_MODULE, not something that's typically lying around in the pserv. | 13:00 |
| allenap | I HATE DJANGO. | 13:00 |
| === flacoste changed the topic of #maas to: 2 weeks until Final Freeze | Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/ | ||
| jtv | There's no way to get the requirements out into... apiclient or something? | 13:09 |
| mgz | allenap: yup, I can upgrade, I just need to deal with scott's change to how cloud-init displays ssh fingerprint stuff | 13:16 |
| rbasak | mgz: will it be an issue if arch constraints start needing to specify "i386/generic" instead of just i386? | 13:18 |
| rbasak | The reason is that armhf/highbank and armhf/armadaxp need to be distinguished | 13:19 |
| mgz | rbasak: wouldn't be an issue, but also not hard to match if only a main(?) arch is given and not a subarch | 13:20 |
| rbasak | mgz: it's a bit annoying as sql queries go, because we concluded above that we want to squeeze both into a single string field | 13:21 |
| mgz | so people could still use arch=amd64 as the constraint, and maas knows that amd64/generic satisfies that | 13:21 |
| rbasak | but we could check for a prefix of arch + '/' | 13:21 |
| mgz | well, for that case you can just expand the input | 13:22 |
| rbasak | it can't be farmed off to the db to process then though | 13:22 |
| mgz | and for arm, you can do much the same, treat "armhf" in the api as filter to ('armhf/highbank', 'armhf/armadaxp') or whatever the acceptable full values are | 13:22 |
| rbasak | Oh, I see | 13:22 |
| mgz | have a dict at the python level of what the short versions are in full | 13:23 |
| rbasak | right | 13:23 |
| roaksoax_ | big/win 11 | 13:40 |
| allenap | jtv: Got time for a review? https://code.launchpad.net/~allenap/maas/no-string-in-mime/+merge/126248 | 14:03 |
| jtv | allenap: right after I answer nature's call | 14:04 |
| allenap | Hah, ta. | 14:04 |
| jtv | allenap: nature and your MP have been answered. | 14:13 |
| allenap | jtv: Thanks :) | 14:15 |
| jtv | Thanks for fixing my test. | 14:16 |
| jtv | hey there matsubara | 14:34 |
| matsubara | hi jtv | 15:01 |
| jtv | matsubara: wanted to give you a heads up for a change I landed: a new maas-provision command that registers a cluster controller with the region controller. | 15:03 |
| jtv | We're not using it yet, but it should work. | 15:03 |
| matsubara | jtv, ok | 15:03 |
| jtv | Assuming that celery is not running, you should be able to start the cluster controller by running "maas-provision start-cluster-controller <region-controller URL>" | 15:03 |
| matsubara | jtv, is this the work to make maas work across different subnet? | 15:04 |
| jtv | Part of it, yes. | 15:04 |
| jtv | It lets us run multiple clusters. | 15:05 |
| jtv | The nice thing is that the exact same registration procedure will work for existing cluster controllers that are rebooting, new ones that haven't been registered yet, and new ones that have just been approved. | 15:05 |
| jtv | Cluster controller asks the region controller: "Hi, I'd like to work for you as a cluster controller. Is that OK?" | 15:06 |
| jtv | Region controller answers with either: | 15:06 |
| jtv | (a) Get lost, creep. I don't know you. | 15:06 |
| jtv | (b) Still waiting for my human to make a decision. Can you call me back in a minute or so? | 15:06 |
| jtv | (c) Sure buddy, and welcome! Here's the information you need to connect to RabbitMQ. | 15:07 |
| jtv | If the cluster was already known, it just immediately gets (c). | 15:07 |
| Daviey | b opens itself to brute force fwiw.. It confirms there is an account with that user. | 15:07 |
| jtv | Daviey: no account or user is implicated. | 15:07 |
| Daviey | so how does the client know the difference between a and b? | 15:08 |
| jtv | Just the http status code in the response. | 15:08 |
| allenap | rvba: Do you have time for a call to help me understand what I'm doing with this dns stuff? | 15:08 |
| rvba | allenap: sure. | 15:08 |
| jtv | matsubara: the important thing to know for the future is that we won't be starting an upstart celery service. We'll be starting this new command, and once it gets a positive response, it will fire up celery. | 15:09 |
| jtv | Once the cluster controller has been accepted, it sends the region controller a request: "now that I'm one of your cluster controller and am connected to the RabbitMQ channel, could you send each of your known cluster controllers their individual credentials etc. over rabbit?" | 15:11 |
| allenap | matsubara: Hey there, can you change the `make check` line in Jenkins to `make offline=true check`? | 15:49 |
| matsubara | allenap, yes, sure | 15:51 |
| allenap | matsubara: Ta. | 15:51 |
| matsubara | allenap, so, should it look like this: make distclean && make offline=true build && make offline=true check? | 15:57 |
| allenap | matsubara: Yep. | 16:21 |
| matsubara | allenap, ok, so it's been done. I take this should be applied for both maas jobs, right? | 16:21 |
| allenap | matsubara: Yes, think so; I backported it to 12.04-nocobbler. | 16:22 |
| matsubara | cool | 16:22 |
| === matsubara is now known as matsubara-lunch | ||
| platypusfriend | Greets | 17:15 |
| platypusfriend | I have a question about configuring maas-dhcp on a system with two network interfaces (I'd like my MAAS server to provision nodes on a private network) | 17:16 |
| platypusfriend | How do I configure maas-dhcp to use my second interface? I wasn't able to find this answer in the documentation, or in a Google search. | 17:17 |
| platypusfriend | Additionally I'm unable to use or install the maas-provision utility. | 17:21 |
| platypusfriend | $ maas-provision generate-dhcp-config -h maas-provision: command not found | 17:22 |
| platypusfriend | $ sudo apt-get install maas-provision Reading package lists... Done Building dependency tree Reading state information... Done maas-provision is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. | 17:22 |
| smoser | roaksoax_, have you poked at all at commissioning ipmi stuff | 17:41 |
| roaksoax_ | smoser: yeah... was just waiting a way to commit that stuff to finish it | 17:41 |
| smoser | ? | 17:42 |
| smoser | can we chat here in 30 minutes? | 17:42 |
| roaksoax_ | smoser: sure | 17:42 |
| platypusfriend | Has anyone been able to use maas-provision with 12.04.1? | 17:43 |
| roaksoax_ | platypusfriend: precise's version of MAAS does not ship maas-provision | 17:51 |
| roaksoax_ | platypusfriend: that's only in quantal | 17:51 |
| platypusfriend | aaahhh.... | 17:51 |
| platypusfriend | ty | 17:51 |
| === matsubara-lunch is now known as matsubara | ||
| === roaksoax_ is now known as roaksoax | ||
| === robbiew1 is now known as robbiew | ||
| roaksoax | wq/win 12 | 22:26 |
| roaksoax | bigjools: around already? | 22:26 |
| bigjools | roaksoax: it seems so | 22:27 |
| bigjools | twins woke up at $EARLY | 22:27 |
| roaksoax | bigjools: hehe sounds like fun | 22:27 |
| bigjools | not so much :) | 22:27 |
| roaksoax | :) | 22:28 |
| roaksoax | bigjools: so do you want to go over the packaging stuff? | 22:28 |
| bigjools | yes gimme 1 min | 22:29 |
| bigjools | roaksoax: ok let's, err, rock :) | 22:35 |
| roaksoax | bigjools: http://paste.ubuntu.com/1227561/ | 22:36 |
| roaksoax | bigjools: ok so overall it is looking good | 22:36 |
| roaksoax | bigjools: now, if you see I've made a few changes. python-* should not really contain any binaries | 22:37 |
| roaksoax | nor config files | 22:37 |
| bigjools | yeah I figured there would be some of that | 22:37 |
| roaksoax | bigjools: now, celery... where is it needed? | 22:37 |
| bigjools | cluster controller | 22:38 |
| bigjools | there's a wrapper that will start it up | 22:38 |
| roaksoax | bigjools: so maas-region-controller does not need it at all? | 22:39 |
| roaksoax | bigjools: nothing else needs the config?" | 22:39 |
| bigjools | nope | 22:39 |
| roaksoax | bigjools: so this changes like: http://paste.ubuntu.com/1227581/ | 22:41 |
| bigjools | why did you make python-maas-provisioningserver conflict with python-django-maas? | 22:41 |
| roaksoax | bigjools: now, maas-dns is to be installed only on the region, on the cluster, or both? | 22:41 |
| bigjools | only region | 22:41 |
| roaksoax | bigjools: check the versions of conflict. Any lower than the current 1049 | 22:42 |
| roaksoax | bigjools: the reason is becuase provisioning server used to be part of python-django-maas and for upgrades to work, we need that | 22:42 |
| bigjools | why does it need that? | 22:42 |
| bigjools | ok | 22:43 |
| roaksoax | ok so maas-dns is *only* on the region | 22:44 |
| bigjools | yup | 22:44 |
| bigjools | roaksoax: argh | 22:44 |
| bigjools | one small fly | 22:44 |
| roaksoax | what is it? | 22:44 |
| bigjools | the region controller does need a celeryd | 22:44 |
| bigjools | let me try and explain | 22:45 |
| bigjools | this is a little awkward and I'm not sure if it's the best way, but we have a concept of a "master" cluster | 22:45 |
| bigjools | where the worker also handles the dns jobs | 22:45 |
| bigjools | it needs to install appropriately | 22:46 |
| roaksoax | bigjools: right, so what's the difference? | 22:46 |
| bigjools | roaksoax: when it sets up the nodegroup it needs to have a special name | 22:46 |
| roaksoax | bigjools: where is this being set? | 22:47 |
| bigjools | so maas generate_dhcp_config | 22:47 |
| bigjools | which maas-dhcp calls right now | 22:47 |
| bigjools | even if dhcp is not managed | 22:47 |
| roaksoax | bigjools: right, so my questions remains, who needs celery? | 22:48 |
| roaksoax | bigjools: python-maas-provisioningserver | 22:48 |
| roaksoax | i pressume | 22:48 |
| bigjools | roaksoax: clusters and the region controller then, sorry I was not clear | 22:49 |
| roaksoax | bigjools: right, but from the clusters and the region, who needs it exactly. Who needs the daemon running | 22:49 |
| bigjools | but there's a requirement for a cluster controller as a "master" | 22:49 |
| bigjools | which is on the region | 22:49 |
| bigjools | roaksoax: let me rewind a bit | 22:50 |
| bigjools | it's not a direct dependency of a region controller, but a region controller needs a master cluster controller somewhere, which needs celeryd | 22:50 |
| bigjools | make sense? | 22:50 |
| roaksoax | bigjools: yes, but my point being is this "maasserver/api.py" imports celery, and then, provisioningserver does | 22:51 |
| bigjools | roaksoax: so a region needs at least one cluster | 22:51 |
| bigjools | roaksoax: yes but the region doesn't run the daemon | 22:51 |
| roaksoax | bigjools: that's what I needed to know | 22:52 |
| roaksoax | bigjools: so cluster runs the daemon, both need python-celery | 22:52 |
| roaksoax | and both need celeryconfig right? | 22:52 |
| bigjools | roaksoax: yes | 22:52 |
| bigjools | roaksoax: however see bug https://bugs.launchpad.net/bugs/1056250 | 22:53 |
| ubot5 | Ubuntu bug 1056250 in MAAS "etc/celeryconfig.py is used by both the region controller code and the worker code." [Critical,Triaged] | 22:53 |
| roaksoax | bigjools: ok so celeryconfig should go in a common package, i.e maas-common and the upstart job with the twisted daemon in maas-cluster-controller | 22:53 |
| bigjools | yes | 22:54 |
| bigjools | well the cluster is a little more complicated | 22:54 |
| bigjools | it needs to start two upstart jobs | 22:54 |
| bigjools | 1. the existing pserv twisted daemon | 22:54 |
| bigjools | 2. the new script that wraps celeryd | 22:54 |
| bigjools | I added the latter in this branch | 22:54 |
| roaksoax | what script?\ | 22:55 |
| roaksoax | bigjools: ok so what else is common? | 22:56 |
| bigjools | maas-cluster-controller.maas-celery.upstart is the new upstart | 22:57 |
| roaksoax | ah ok | 22:57 |
| roaksoax | that's fine then | 22:57 |
| * bigjools thinks what else is common | 22:57 | |
| roaksoax | so we only need the celeryconfig and maas_local_celeryconfig.py in a common package | 22:57 |
| bigjools | yeah this is what I thought could go in the provisioningserver package :) | 22:57 |
| bigjools | but you nixed that | 22:57 |
| roaksoax | bigjools: yeah python-xyz packages usually don't contain config files | 22:59 |
| roaksoax | at least i can't recall seeing one | 22:59 |
| bigjools | I learned something today | 22:59 |
| roaksoax | bigjools: ok, now DHCP only on cluster | 23:00 |
| roaksoax | right? | 23:00 |
| bigjools | yep | 23:01 |
| roaksoax | bigjools: now, rsyslog.d, where will it be? | 23:01 |
| bigjools | roaksoax: NFI! | 23:01 |
| bigjools | that one I didn't play with really, needed to talk about it | 23:01 |
| roaksoax | ok that's something to fiure out | 23:01 |
| bigjools | it's for syslog from the nodes right? | 23:01 |
| roaksoax | bigjools: now, preseeds? | 23:01 |
| roaksoax | bigjools: yes | 23:02 |
| bigjools | depends where we want the log to go then, region or cluster? | 23:02 |
| bigjools | logs* | 23:02 |
| roaksoax | bigjools: let's take care of other stuff first | 23:02 |
| * bigjools spins the quantal roulette wheel daily updates | 23:02 | |
| bigjools | preseeds are on region only | 23:02 |
| roaksoax | bigjools: so the cluster will provide DHCP, tftp, right? | 23:03 |
| roaksoax | bigjools: hold on... so when a node boots with the cluste,r the cluster will tell him "here's your preseed in the REGION?" | 23:03 |
| bigjools | roaksoax: no, the cluster controller fetches them over the api | 23:04 |
| roaksoax | bigjools: ok, what about squid-deb-proxy? | 23:04 |
| roaksoax | region too? | 23:04 |
| bigjools | yeah | 23:04 |
| bigjools | that one's a bit horrible | 23:04 |
| roaksoax | ok so I guess syslog also makes sense to be in region for now | 23:04 |
| bigjools | it is not scalable | 23:04 |
| roaksoax | bigjools: well TBH, that's why mirrors are for :) | 23:05 |
| roaksoax | bigjools: now there's a problem | 23:08 |
| bigjools | /o\ | 23:08 |
| roaksoax | bigjools: the problem is that in packagin we modify the config files in order to set the settings we want automatically right? | 23:08 |
| roaksoax | bigjools: so everytime we upgrade, and the upstream config file differs from the installed one (pre-modification), it will prompt | 23:09 |
| roaksoax | the user | 23:09 |
| roaksoax | to upgrade it | 23:09 |
| bigjools | yeah | 23:09 |
| roaksoax | so we need to find a way to handle that gracefully | 23:09 |
| bigjools | and that is very very annoying | 23:09 |
| roaksoax | indeed | 23:10 |
| * bigjools stares at cups packaging | 23:10 | |
| roaksoax | and annother annoring thing is dbconfig-common | 23:10 |
| roaksoax | bigjools: ok so thjis is what I've got so far: http://paste.ubuntu.com/1227614/ | 23:11 |
| roaksoax | bigjools: it would be usefull to have this branch in ~maas-maintainers so I can alsocommit to it | 23:11 |
| bigjools | roaksoax: let me work something out | 23:12 |
| bigjools | roaksoax: alternatively I can merge your branch if you have a copy | 23:12 |
| bigjools | prob easier in the long run | 23:12 |
| roaksoax | bigjools: apply this patch so far :) http://paste.ubuntu.com/1227614/ | 23:12 |
| roaksoax | take a look at it, and if you agree please merge it | 23:12 |
| bigjools | hax0ring without a branch? :) | 23:12 |
| bigjools | oik | 23:12 |
| bigjools | ok even | 23:13 |
| bigjools | roaksoax: ok changes pushed up, the diff will update soon | 23:15 |
| roaksoax | bigjools: what comes to my head is find a way to determine when it has changed upstream automatically, and when that happens, simply move the old config file, display a message, and reconfigure things | 23:15 |
| bigjools | roaksoax: should we just write the new config to the side and leave a message? | 23:16 |
| bigjools | let the user merge later | 23:16 |
| bigjools | Ideally we won't amend the *local* configs | 23:16 |
| roaksoax | bigjools: right, but pserv.yaml and txlongpoll.yaml have no local configs | 23:17 |
| bigjools | roaksoax: also see https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1055951 | 23:17 |
| ubot5 | Ubuntu bug 1055951 in maas (Ubuntu) "maas-dhcp has no sudoers permission to start maas-dhcp-server" [Undecided,New] | 23:17 |
| bigjools | yeah | 23:17 |
| roaksoax | bigjools: and maas_local_settings has changed quite frequently | 23:17 |
| bigjools | it has, until it settles down :) | 23:17 |
| roaksoax | bigjools: the fix for the bug above: https://code.launchpad.net/~andreserl/maas/packaging_update | 23:20 |
| roaksoax | https://code.launchpad.net/~andreserl/maas/packaging_update/+merge/126350 | 23:21 |
| bigjools | roaksoax: that was quicj! | 23:21 |
| bigjools | roaksoax: can you set a commit msgt | 23:21 |
| bigjools | msg | 23:21 |
| roaksoax | :) | 23:21 |
| roaksoax | done | 23:22 |
| roaksoax | alright, i'm off | 23:22 |
| roaksoax | will probably be back later | 23:22 |
| bigjools | roaksoax: nice one, thanks. One more problem is that of upgrading from 12.04 | 23:22 |
| bigjools | the upgrade scripts are in maas-region-controller | 23:22 |
| bigjools | but will be checking unknown versions | 23:22 |
| bigjools | since they used to be in maas | 23:22 |
| bigjools | but we can chat when you get back | 23:23 |
| roaksoax | alrighty. Will talk about that later | 23:25 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!