#maas 2013-08-05
<AskUbuntu> how do i apply the maas bug fix? | http://askubuntu.com/q/328858
<AskUbuntu> Juju zookeeper stuck at 1st bootstrap in maas enviornment | http://askubuntu.com/q/329024
#maas 2013-08-06
<roaksoax> bigjools: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1208497
<ubot5> Ubuntu bug 1208497 in MAAS "netboot flag defaults to 'true' on upgrade, even for allocated nodes" [Critical,Triaged]
<bigjools> I saw
<roaksoax> ok;)
<bigjools> :)
<roaksoax> then i xan go sleep
<roaksoax> lol
<bigjools> waiting for my Django expert to arrive before I decide what to do about it
<roaksoax> have a good one man
<bigjools> do sleep :)
<kentb> If I run a juju terminate-machine on a maas node to return it to a ready state, can I re-use it to deploy another charm on it?  If so, do I need to re-PXE boot it?  deleting the node, re-enlisting, re-commissioning seems like overkill...
<rbasak> kentb: you shouldn't have to do that. I think juju deploy should just work.
<kentb> rbasak: ok.
<rbasak> And MAAS will re-PXE boot it, reinstall Ubuntu and the service unit should just come up.
<kentb> rbasak: ok. got it.
<rbasak> That's only my understanding though.
<kentb> makes sense, though
#maas 2013-08-07
<theJujuGuy> hey! I use MaaS with the MaaS-dhcpd-Server. I want that the dhcp-server serve DHCP-Request on interface eth0 and eth1. I add the INTERFACES="eth0 eth1" to /etc/default/isc-dhcp-server, but this dont effect the init-script using eth0 and eth1.
<theJujuGuy> sorry i must go. see you next week.
<theJujuGuy> hey! I use MaaS with the MaaS-dhcpd-Server. I want that the dhcp-server serve DHCP-Request on interface eth0 and eth1. I add the INTERFACES="eth0 eth1" to /etc/default/isc-dhcp-server, but this dont effect the init-script using eth0 and eth1.
<theJujuGuy> problem solved: ok i found the /etc/init/maas-dhcp-server.conf
<theJujuGuy> anyone use vmware virtual machines with maas? i want to use the vsphere web api to power up/down the nodes. can i use the virsh?
<jreingol> question...
<jreingol> im trying to get juju working with maas and it keeps reporting that the provider-state map being returned by juju is nil
<jreingol> how might one fix said problem?
#maas 2013-08-08
<freeflying> http://paste.ubuntu.com/5962495/ anyone had this issue before NodesNotAvailable: No matching node is available. but actually lots of machines listed as ready in maas
<rbasak> freeflying: bug 1064291 maybe?
<ubot5> bug 1064291 in juju "Default constraints make no sense on MAAS" [Undecided,New] https://launchpad.net/bugs/1064291
<rbasak> (workaround available)
<freeflying> rbasak: sounds abit different, this maas environment worked, but after I bootstrap with constraints, then ran into the error I mentioned avove
<freeflying> above
<rbasak> freeflying: if you're bootstrapping with contraints, then the error means that no node matched your specified constraints, surely?
<freeflying> rbasak: I could bootstrap with constraint, which was maas-name, but not any other instance after bootstrap
<freeflying> rbasak: its repeatable with destroy-envirment
<rbasak> freeflying: I think a constraint on bootstrap does something you don't expect. All other nodes you attempt to start will have the same constraint applied, and of course you can only have one node work with that constraint. Could this be your problem?
<rbasak> freeflying: if that's what it is, then I agree it's confusing, but juju developers disagreed. I think it's in a bug somewhere.
<freeflying> rbasak: I understand, the issue is still there even I bootstrap without constraints
<rbasak> freeflying: I'm not sure then, sorry. You can see what constraints juju asked MAAS for in the logs somewhere I think. Maybe the web server ones?
<freeflying> rbasak: all the nodes enlisted to maas has same spec, 1024M ram, 1 vcpus, 2 nics, 2 block disk
<freeflying> rbasak: and now I'm rebootstraping, node was allocated without problem
<rbasak> freeflying: I still think the problem is at the juju end. MAAS just uses the contraints it has been given, and I've never seen it fail to work. The problem's always been that the juju end has specified constraints that MAAS cannot fulfil, even if that's not what the juju user intended.
<freeflying> rbasak:  are you referring to python version juju or it still function in this way in go version?
<rbasak> freeflying: I know about the python version. I don't know what the go version does.
<freeflying> rbasak: thanks a lot, I'm thinking about filing a bug against :)
<AskUbuntu> juju zookeeper installation stuck | http://askubuntu.com/q/330295
#maas 2013-08-09
<mwhudson> bigjools: hello
<bigjools> mwhudson: good day to you sir
<mwhudson> bigjools: i have a question about the many versions of maas apparently floating around
<mwhudson> in particular people keep running into this bug https://bugs.launchpad.net/maas/+bug/1172966
 * bigjools cowers
<ubot5> Launchpad bug 1172966 in MAAS 1.2 "SAY command in config.local.template breaks local boot on highbank" [High,Fix committed]
<mwhudson> even though it's been fixed at least twice...
<mwhudson> apparently it's still in the version of maas you get if you say "apt-get install maas" on raring
<bigjools> 1.2 is the version in precise
<mwhudson> is that a surprise?
<bigjools> you need to hassle the packagers, I am just the poor upstream
 * bigjools stares at roaksoax :)
<mwhudson> ok
<mwhudson> i remembered something about the raring version of maas having fpi support in it which isn't in trunk or something like that...
<bigjools> no, FPI is not done anywhere
<bigjools> I am goinbg to be pushing hard for it in 13.10 though
<bigjools> since not having it makes development cycles frustrating
<mwhudson> aah ok
<mwhudson> glad i was making that bit up
<bigjools> mwhudson: FWIW there are PPAs with fixes in if desperate
<mwhudson> bigjools: i'm not really desperate i just want people to stop hitting the bug over & over again
<mwhudson> bigjools: i guess
<mwhudson> bigjools: would you advise people to use the ppas by default?  i guess i would, at least for armhf
<bigjools> mwhudson: not by default, we don't offer any support
<bigjools> unless there's a specific thing we recommened using them for
<mwhudson> bigjools: so i should beat up roaksoax when he's here?
<bigjools> pretty much :)
<mwhudson> to get the fix SRUed
<mwhudson> ok
<mwhudson> i guess saucy has a newer version anyway?
<bigjools> yeah that's mostly up to date
<mwhudson> bigjools: which ppas have the fix?
<bigjools> mwhudson: daily ones at least
<bigjools> check the package versions' bzr string
<kurt_> Hi - does anyone know what this error is about? OAUnauthorized
<kurt_> http://pastebin.com/u45kvUGb
#maas 2013-08-10
<mattrae_> kurt_: hey when i've seen OAuthUnathorized, usually its because the time is out of sync between the node and the maas server
<mattrae_> kurt_: because oauth doesn't like time being out of sync
<mattrae_> kurt_: are you seeing that error when you try to enlist a node in maas?
<kurt_> mattra: I've restarted the complete enlistment process via maas.  The time was definitely off.  But I'm not sure how to correct that in the individual instances.
<kurt_> mattrae rather
<kurt_> Hi - I'm using vmware / virtual machines with maas.  It's worked fairly well so far.  But I am seeing a ton of "No PXE template found" errors in the OOPS log.  Any idea how to fix these?
<kurt_> http://pastebin.com/mjA2VEBL
<kurt_> Also I'm having a problem trying to deploy juju-gui.  While I know that kind of problem should be discussed in the juju list, the problem seems to be with maas and its ability to spin up a node.  There are OAuthUnauthorized errors in the maas.log.  This isn't a time issue.  At least the control node and the first node allocated to root are in sync w/r time.
<kurt_> http://pastebin.com/ud3ErH4g
#maas 2014-08-04
<bigjools> allenap or gmb, easy review for you! https://code.launchpad.net/~julian-edwards/maas/optional-ip/+merge/229439
#maas 2014-08-05
<Spads> I've just upgraded some MAAS infrastructure from Precise and 1.4 to Trusty and 1.5, and I notice that one of my cluster controllers is running low on disk due to /var/lib/maas/ephemeral being full of old data.  Everything's from months ago, though the Web UI says it has up to date images (which I can find elsewhere).  Is it safe for me to blow away /var/lib/maas/ephemeral after my 1.5 upgrade?
<onicrom> hello everyone
<Spads> The cluster controllers in MAAS 1.5 seem to be trying to connect to two high-numbered ports on the region controller.  Apache is listening on those ports on the region controller, but I can't find the configuration that sends them there.  What are these ports, and are they predictable?
<onicrom> are the files in /etc/maas/templates/pxe essentially pxelinux.cfg/syslinux.cfg files?
<onicrom> and is it safe to edit them (once you copy/rename?)
<jhobbs> onicrom: yes, they are templates for generating files like pxelinux.cfg files
<jhobbs> onicrom: what are you trying to get done by copying/editing them?
<onicrom> i was looking to add "CONSOLE 0" and "SERIAL 1 115200 0"
<jhobbs> onicrom: have a look at http://maas.ubuntu.com/docs/kernel-options.html
<onicrom> ive appended console=tty0 console=ttyS1,115200n8 to that
<onicrom> wasn't sure if the actual pxeboot section
<jhobbs> oh - those are pxelinux directives?
<onicrom> would be sent to console, or just the install part
<onicrom> correct
<jhobbs> so you can see pxelinux output on the serial console?
<onicrom> thats the goal
<jhobbs> if it's pxelinux stuff you should be able to add it in the file wherever you would in a normal pxelinux file
<jhobbs> was copying/renaming just to get a backup?
<onicrom> well i wasnt sure if i needed to rename either the config.commissioning.template to config.commissioning
<onicrom> or edit the template
<jhobbs> ahh yeah just edit the template
<onicrom> or edit the arch specific file
<jhobbs> maas opens the template and renders it and then sends it off via tftp
<onicrom> ok cool
<onicrom> so would it be best to add to config.install.tem... config.commissioning.tem... or config.local.amd64.tem....
<jhobbs> they're for different stages
<jhobbs> if you want pxelinux output all the time, do all of them
<onicrom> would it be correct in assuming editing the amd64 would be invoked at all stages
<onicrom> assuming you are using that arch
<jhobbs> config.local.amd64.template is only for localbooting, after an install is complete
<jhobbs> otherwise amd64 uses the generic ones without arch names in the files
<onicrom> got it
<jhobbs> they're not hierarchical so you have to edit them all
<onicrom> thanks much
<jhobbs> you're welcome
<onicrom> config.local and config.local.arch need to be changed then?
<jhobbs> hmmm
<jhobbs> i'm not sure why there are both there
<jhobbs> couldn't hurt :)
<onicrom> do you work on the project?
<jhobbs> yes
<onicrom> might i suggest something... the .template extension makes me think these files , should you need to change them, should be renamed removing the .template
<onicrom> or copy, without .template and make changes, the system reads the non.template files should they exist
<onicrom> (dont mean to complain just how i assume things to work on other tools)
<jhobbs> i could see how that's confusing
<jhobbs> feel free to file a bug on https://bugs.launchpad.net/maas
<jhobbs> best to have stories from real users for that sort of thing
<onicrom> i wouldnt vcall it a bug :)
<onicrom> but sure thing
<onicrom> i have another question :) around preseed
<jhobbs> ok
 * jhobbs in and out, just ask and i'll try to answer
<onicrom> https://bugs.launchpad.net/maas/+bug/1352938
<ubot5> Launchpad bug 1352938 in MAAS "the use of the file extension .template can be confusing" [Undecided,New]
<onicrom> bug filed :)
<jhobbs> cool, thanks
<onicrom> jhobbs: i'm not familair with preseed at all so this is going to be amateur hour, under /etc/maas/preseeds what are the purposes for the various files
<jhobbs> onicrom: have you read this? https://maas.ubuntu.com/docs/development/preseeds.html
<jhobbs> onicrom: preseeds for maas either generate cloud-init, curtin, or debian-installer config, depending on which state/install type is being used
<jhobbs> onicrom: users can use them to customize enlistment/commissioning/installation
<jhobbs> they're template files again, just like the pxe ones
<onicrom> cool
<onicrom> i read references to the debian-installer but my maas only says default or fast install
<onicrom> i assume default is debian installer?
<jhobbs> yeah
<onicrom> woah i missed this doc entirely somehow :)
<onicrom> thanks! also... the terms commission,enlist etc are these defined somwhere?
<jhobbs> i can't find anything
<jhobbs> that sucks
<jhobbs> enlistment makes maas aware of the node, commissioning does some hardware inventory
<jhobbs> gotta run for a bit
<onicrom> any takes on 'curtin'
<jhobbs> onicrom: it's the fastpath installer
<jhobbs> onicrom: it's default in newer maas's - it's all i ever use
<onicrom> intttersting
<onicrom> how does customization work in this new fancy faster method
<rbasak> It's designed to be very customisable
<rbasak> See: http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/doc/topics/overview.rst
<rbasak> You can pretty much replace everything.
<onicrom> cool i shall readup
<onicrom> does 14.04's maas pkg get upgraded as releases go stable (1.5->1.6) or is that a more manual process?
<onicrom> by 'get upgraded' i mean apt-get upgrade or dist-upgrade if more appropriate
<jhobbs> you can apt-get upgrade mas
<jhobbs> *maas
<jhobbs> it will pick up whatever the latest release for trusty is
<jhobbs> as long as that's what you're running
<onicrom> will trusty upgrade to mewer point releases?
<onicrom> rather... be upgraded to.
<jhobbs> hmm i think roadmap questions are above my paygrade
<jhobbs> you can always pin to a newer version if you want to though
<onicrom> nods
<onicrom> btw
<onicrom> damn fine job on this thing
<onicrom> ive setup my fair share of build systems from scratch with jump/kickstart
<onicrom> all the annoying stuff is taken care of, just need to sort out the customizations :)
<jhobbs> glad you like it and thanks for the feedback!
<jhobbs> bugs on any issues or annoyances you run into are welcome
<onicrom> well i have 2 data centers im building out (~40 servers) so im going to be diving real deep :)
<onicrom> does maas read all files in /etc/maas/preseeds? ... if i 'backup' the files in /etc/maas/preseeds (as in /etc/maas/preseeds/preseed_master.dist) will it do badness?
<onicrom> looks like the installer output is not being sent to my serial console .. the commissioning is sent to serial console but the install is not
<onicrom> hrms
<newell> onicrom, have you been checking the logs in /var/log/maas?
<onicrom> now its hung
<onicrom> checking logs thanks :)
<newell> np
<onicrom> <bitching> damn dell drac seems to drop offline whenever the ubuntu installed 'probes' the network devices </bitching>
#maas 2014-08-06
<gmb> allenap, rvba`, bigjools Awright?
<gmb> Made it to internet at last.
<allenap> gmb: Hello!
<gmb> Hola
<gmb> allenap: Do you need any help with the celery->RPC work?
<gmb> Or rather: I'm free, Mr Humphries; what can I help with?
 * gmb -> Location switch; bbiab
<gmb> allenap: Iâd like to help with the RPC work; is there any reason I canât start with the âperiodically report boot imagesâ task?
<gmb> (although I have no idea how to do peridoic RPC tasks without celerybeat)
<onicrom> question... im seeing the following error in my maas.log
<onicrom> ERROR 2014-08-06 13:08:35,490 maasserver ################################ Exception: {u'mac_addresses': [u'One or more MAC addresses is invalid.']} ################################
<onicrom> not sure how it could be invalid, it pxe booted and its the actual mac
<onicrom> i think i figured it out
<onicrom> maas is expecting the mac to have upper case letters.
<onicrom> (which is not that obvious)
<allenap> gmb: Sure, thatâs a good one.
<allenap> onicrom: That sounds like a bug; MAAS should allow both, or at least normalise automatically. Iâll file a bug.
<allenap> onicrom: Was there any more detail to that error message?
<onicrom> filed
<allenap> onicrom: Thanks! Iâd just seen that seconds ago.
<onicrom> allenap: its weird when i entered it using all caps the comissioning would fail
<onicrom> when i entered it lowercase
<onicrom> the maas.log would error
<onicrom> the UI seems to display lowercase either way
<onicrom> im now running into a weird issue with the dell.. i use serial over lan and as soon as the install gets to a certain point the drac remote management ip becomes unresponsive until i reboot the machine
<allenap> blake_r, newell, jhobbs: ^
<newell> allenap, howdy
<onicrom> also unlreated question... what is maas using iscsi for?
<newell> allenap, do you have  bug number?
<newell> actually nm, I will look on launchpad
<allenap> newell: I was pointing you to the Dell issue onicrom is having.
<newell> yeah I see the bug now
<newell> oh so not the bug?
<newell> okay
<onicrom> (im less concerned about the bug i filed as i figured out the nuance)
<onicrom> i think it might have something to do with my preseed file as i started messing with the storage... but in either case i dont see why the dell's management interface becomes unresponse :(
<jhobbs> onicrom: do you have to reboot the system manually?
<newell> onicrom, can you pastebin your console output?
<onicrom> jhobbs: i have a kvm remote console which i can three finger solute
<jhobbs> onicrom: it should reboot automatically after an install
<onicrom> which works and the DRAC responds as soon as the reboot is initiated
<onicrom> jhobbs: i doesnt actually start the install actually
<onicrom> it get stuck at the kernl booting screen before the install is initiated.
<jhobbs> hmm
<onicrom> just happened again :( appears to be unlreated to my preseed as i restored the default
<onicrom> ooh weird.... the dhcp ip is pingable
<jhobbs> are you passing serial console parameters on the kernel command line?
<onicrom> but my console (kvm and serial) has stopping printing
<onicrom> u am
<onicrom> *i am
<onicrom> but i have remote kvm too so i can watch watch would be on a monitor
<jhobbs> console=ttyS0 and console=tty0 too?
<onicrom> yes
<onicrom> console=tty0 console=ttyS1,115200n8
<newell> onicrom, anytime I have run into that issue it is because I wasn't passing the correct kernel parameters that jhobbs is mentioning
<onicrom> to be exact
<jhobbs> how are you passing them?
<onicrom> in the global kernel params
<onicrom> i see all the comissioning happen
<jhobbs> ah ok
<onicrom> via console (and tty0)
<jhobbs> are you doing a fastpath install or a debian installer install?
<jhobbs> if it's fastpath, it's the same kernel/initrd as commissioning uses
<onicrom> debian
<jhobbs> it'd be worth trying fastpath to see if it's specific to the d-i kernel
<onicrom> i can certainly give that a try i want to see if the install actually works (i probably botched the last presee file)
<jhobbs> k
<onicrom> the machine is still pingable on its actual interface the drac is still offline :(
<onicrom> ive noticed that with cd based installs too
<onicrom> when it goes to 'discover' the network devices
<jhobbs> we test with a lot of dell/drac systems on a very regular basis
<onicrom> drac management interface goes offline
<jhobbs> i don't remember seeing that kind of behavior before
<onicrom> i am using the ghetto drac6e
<onicrom> so its using a 'shared' interface it doesnt have a dedicated nic
<jhobbs> ahh
<jhobbs> sound ripe for kernel/firmware interaction bugs
<onicrom> ineed
<onicrom> *indeed
<onicrom> i cant complain tooo much
<onicrom> x5670 x 2 96GB ram 6x600G drives with bbu raid card
<onicrom> for like 3k
<onicrom> i bought 20 of them for this openstack poc :)
<onicrom> i mean i can always complain haha
<onicrom> i digress... i am still at a loss as to why my rmeote kvm is stuck at a kernel boot message though
<jhobbs> is the kvm via drac or separte?
<jhobbs> separate
<jhobbs> if you're seeing stuff in CD installs too it's likely not a MAAS specific issue though
<jhobbs> you might try #ubuntu-kernel or https://bugs.launchpad.net/ubuntu/+source/linux/+bugs for those
<onicrom> kvm is separate
<jhobbs> does pxelinux print the kernel command line parameters for you prior to trying to boot linux?
<jhobbs> might be worth double checking that console=tty0 is still there
<onicrom> it did
<onicrom> i saw it in both
<jhobbs> dang
<onicrom> the host is still not booting but its in some state...
<onicrom> i cant ssh in (con refused)
<onicrom> drac not reachable and kvm shows boot message that is stale...
<onicrom> id really like to see wtf is going on lol
<onicrom> the install should ber finished but its possible its still going i gues...
<jhobbs> ssh would be up if the install finished, the system would reboot afterwards
<jhobbs> but then it wouldn't have an IP if it hung at early linux boot
<jhobbs> wouldn't respond to pings i mean
<onicrom> a reboot would re-enable drac
<onicrom> exactly :) its in some weird state
<jhobbs> you could try adding earlyprintk to the command line parameters
<jhobbs> but.. if it gets far enough to get an ip that's past earlyprintk time
<onicrom> btw... what is iscsi used for?
<jhobbs> "ephemeral" booting
<jhobbs> during enlistment, commissioning, install, nodes boot up via pxe and then mount iscsi for their root filesystem, with a ram only overlay
<jhobbs> fastpath install that is, not d-i
<onicrom> what about d-i?
<jhobbs> d-i is all contained in the initrd
<onicrom> gotcha
<onicrom> ok i will try fast path on another node which i have kvm/serial (i only have 4 devices on kvm the rest are serial only :()
<jhobbs> hopefully that works, it's a lot better imo anyhow
<onicrom> i need to figure out how to customize that install :)
<jhobbs> yeah it's something new to learn, but has the huge advantage of having a rich ubuntu environment during install
<onicrom> well...i dont know the debian installer either
<jhobbs> d-i is on top of busybox
<onicrom> i was starting to play with that i have no bias
<jhobbs> no python, not even perl, i think you don't even get awk
<jhobbs> maybe sed
<onicrom> ok fingers crossed building another box with fast now
<jhobbs> well good luck, i'm off for the night
<onicrom> wooo
<onicrom> jhobbs: its working!
<onicrom> any docs around on customizing the install via fastpath
<rbasak> onicrom: http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/doc/topics/overview.rst
<onicrom> thanks rbasak
<lamont> what exactly is maas telling me to fix when it says: Unable to get RPC connection for cluster 'master'
<bigjools> lamont: clusters and region appserver threads attempt to maintain bidirectional connections, without which maas won't work properly. It's possible you are firewalled since they currently use high-numbered ports for this.
<lamont> who creates the connection, I wonder?
<onicrom> rbasak: im looking to create a customized partition table, are there more specific exaples to configure curtin
#maas 2014-08-07
<onicrom> morning folks
<allenap> Morning onicrom.
<onicrom> is there an upgrade guide for maas? or if i pin in to a newer version and apt-get install would that 'break' things?
<blake_r> allenap: could I get review of https://code.launchpad.net/~blake-rouse/maas/large-object-store/+merge/229938
<allenap> blake_r: Sure, Iâll take a look :)
<blake_r> allenap: thanks
<onicrom> #ubuntu-kernel is so quiet
<blake_r> rvba`: could you review this https://code.launchpad.net/~blake-rouse/maas/largefile-model/+merge/229949
<jhobbs> i think rvba is on vacation
<blake_r> oh okay
<blake_r> jhobbs: maybe you then?
<roaksoax> blake_r: he is
<jhobbs> i'm looking now blake_r but i think stuff stuff should go through core team too
<jhobbs> stuff stuff
<allenap> blake_r: Sorry for a very slow review; Iâve been dealing with a terrible builder via email.
<blake_r> allenap: haha I remember you telling me about your builder
<blake_r> jhobbs: nah, you good enough for me!
<blake_r> ;-)
<jhobbs> blake_r: done, have some questions inline
<onicrom> are there examples of how one customizes the partition layout using fastpath installer?
<onicrom> does curtin execute everything that's in preseed_master as well
<onicrom> hrms found some stuf fin /usr/lib/curtin/helpers...
<onicrom> we really need more documentation around curtin
<allenap> blake_r: Iâm looking at large-object-store again. Just waiting for Launchpad to scan the branch.
<blake_r> allenap: thansk for the review
<allenap> blake_r: Youâre welcome.
<blake_r> jhobbs: fixed the branch you reviewed, if you can give it another look
<onicrom> anyone have an issue with a newly installed system being unable to apt-get update because its looking to download files named Packages instead of Packages.gz or Sources/Sources.gz?
<onicrom> i think i found the problem..../etc/apt/apt.conf.d/90curtin-aptproxy
<onicrom> Acquire::HTTP::Proxy "http://10.211.25.101:8000/";
<blake_r> onicrom: there is a squid-deb-prxoy running on the maas server
<onicrom> just found that
<onicrom> it seems to be doing something to the apt-get update request
<onicrom> which stops it from downloading the Packages.gz/bz2 file and makes it look for an uncompressed file
<blake_r> onicrom: don't know much about squid-deb-proxy but that seems wierd
<onicrom> refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0
<onicrom> yyyup
<onicrom> its a bug...
<onicrom> it you change  Main archive (required) , you need to update /etc/squid-deb-proxy/mirror-dstdomain.acl
<onicrom> and restart the squid proxy
<lamont> following along with http://maas.ubuntu.com/docs1.5/install.html#post-install - I do the createadmin, and then I say "maas my-maas-session node-groups import-boot-images" and it says: /usr/lib/python2.7/dist-packages/maascli/__main__.py: error: argument COMMAND: invalid choice: u'my-maas-session' (choose from u'list', u'login', u'logout', u'refresh', u'root')
<lamont> 1.5.2+bzr2282-0ubuntu0.2~ctools0
<lamont> what's the right command there?
<lamont> http://paste.ubuntu.com/7982829/ <-- that's what maas-import-pxe-files gives me
<lamont> ah, s/my-maas-session/root in my case
#maas 2014-08-08
<jason___> anybody have a few minutes to point me in the right direction?
<mwhudson> jason___: depends. where are you trying to get to?
<jason___> trying to get farther then a pxe boot and get them to install and work on an HP c7000
<jason___> still digging through all the info i can find :P
<jtv> bigjools: do you remember what we decided about the API problem of "PUT one field, reset all the others"?  Do we simply accept that clients have to read and re-PUT the full object?
<bigjools> jtv: I don't remember if we even decided anything
<bigjools> I think PUTing the whole thing makes sense
<jtv> For some value of sense.
<bigjools> since you can't create that way
<bigjools> but I expect we have examples where that's not the case
<bigjools> jtv: if you want a break from coding I have a lovely branch up for review :)
<jtv> Coding?  I wish.
<lifeless> I knew that word once
<roaksoax> jtv: you scheduled the IPv6 meeting for FRiday next week :)
<roaksoax> 344
<roaksoax> err
<jtv> Argh!
 * jtv heads back to  Google
<jtv> And the wrong timezone, too...
<onicrom> morning folks
<jtv> Hi onicrom
<onicrom> fingers crossed i can get this 'custom' partition working in curtin
<onicrom> ok i seem to have found whats going on
<onicrom> it looks like the default for my system is pt_mbr
<onicrom> which has a bug
<onicrom> it also looks like the params passed to pt_mbr are ignored.
<onicrom> also a bug
<onicrom> ok submitted a bug
<roaksoax> /query/win 16
<roaksoax> err
<allenap> Does anyone have time for a shortish review? https://code.launchpad.net/~allenap/maas/rpc-alt-cluster-rpc-fixture/+merge/230109
<blake_r> allenap: done
<allenap> blake_r: Thanks!
<blake_r> allenap: https://code.launchpad.net/~blake-rouse/maas/add-largefile-migration-fix/+merge/230129
<blake_r> allenap: could I get a really quick review, for my stupidity!
<allenap> blake_r: Sure.
<allenap> blake_r: Is IntegerField really not enough? Woo.
<blake_r> allenap: yeah it is, but might not be if file is larger than 4tb, that should never happen!
<blake_r> allenap: sorry I mean large than 2gb
<blake_r> allenap: you set it to approved? not you approving it
<allenap> blake_r: Sorry! Fixed.
<blake_r> allenap: thanks
<jtv> allenap: my branch may have a huge diff but it's actually really really simple: moved a bunch of functions from provisioningserver.utils to provisioningserver.utils.twisted.
<onicrom> ok i need to speak to the guy who wrote curtin :)
<onicrom> i have hacked the /usr/lib/curtin/helper/common script 6 ways from sunday
<onicrom> i can see that my chages are being executed, but the installed system always always creates a 2tb root filesystem
<jtv> I think that's smoser.
<jtv> blake_r, did you say you figured out how to make curtin sing & dance?
<blake_r> jtv: I stay far away from the bash scripting that smoser wrote!
<jtv> Good man.  But _someone_ said they figured it out and it wasn't so hard once you cracked the secret.
<jtv> jhobbs maybe?
<blake_r> jtv: i wrote the uefi partitioner in the bash
<blake_r> jtv: i cried most of the time, lol
<jtv> (Still think they should have named the GNU shell "gnash")
<Spads> that's the flash player
<jtv> It is _now_, yes.  But at the time there wasn't one.  :)
<blake_r> onicrom: if you have a paste of your bash script I can take a look
<onicrom> blake im updating the bug
<onicrom> blake_r: https://bugs.launchpad.net/maas/+bug/1354426
<ubot5> Ubuntu bug 1354426 in MAAS "curtin creates a 2TB root fs/partition by default" [Undecided,New]
<onicrom> thanks ubot5 :)
<blake_r> onicrom: you want the partition bigger than 2tb?
<blake_r> onicrom: you cannot create a partition larger than 2tb on mbr partition layout
<blake_r> onicrom: you need to use a gpt partition layout
<onicrom> i know
<onicrom> i dont want to
<onicrom> i want to create a 50Gb root partition
<onicrom> the default option in common is to use mbr, i tried to switch this to gpt and modify the size values there but that just errored completely the pt_gpt function is broken
<blake_r> onicrom: are you sure that '1,6490,L' is correct?
<onicrom> based on the sfdisk man page yes
<onicrom> ive tried using cylendars sectors etc
<onicrom> same thing
<onicrom> based on some math that is the correct number of cyls for 50gb
<onicrom> or ~ anyway im not that picky
<blake_r> looks like its only 50gb partition, thats what you want, you just need another line for the other partition
<onicrom> what other partition?
<onicrom> no matter what i do something is creating a 2TB partition
<onicrom> so weird doesnt matter what i execute something creates a 2TB partition... i wonder... let me try a brand new machine without any existing table maybe the bit that wipes the existing partitions is broken :)
<jtv> blake_r: have you run the 1.6 (or 1.5, 1.4 etc.) tests lately?  I'm seeing a test failure in trunk that's related to commissioning timeouts, and it looks like it might be timezone-sensitive.
<jtv> You're in a negative UTC offset now, right?
<blake_r> jtv: unit tests? i am running trunk
<blake_r> jtv: never had an issue
<jtv> Right, unit tests â specifically, src/maasserver/tests/test_api_commissioning.py:TestCommissioningTimeout
<blake_r> jtv: i am -5
<blake_r> jtv: never seen a failure there
<jtv> Yes, if this is the timezones, then trunk should pass for you â but maybe you'd get the converse bug in 1.6 or older.
<blake_r> jtv: i will try 1.6
<jtv> Where the tests creates a node that started commissioning an hour and one second ago, but it doesn't time out yet.
<jtv> Thanks!
<jtv> You'd have to run "make && ./bin/test.maas src/maasserver/tests/test_api_commissioning.py:TestCommissioningTimeout"
<jtv> (I filed this as bug 1354511...)
<ubot5> bug 1354511 in MAAS "check_commissioning fails test: test_check_with_commissioning_but_not_expired_node" [Critical,Triaged] https://launchpad.net/bugs/1354511
<blake_r> jtv: all 3 passed for me
<jtv> Grrrrr
<jtv> Thanks.
<blake_r> Your welcome.
<jtv> blake_r: I think I'm one step closer!  My databases in 1.6 and trunk seem to be running at different timezone offsets...
<jtv> It may have been the travel that caused it.
<blake_r> jtv: isn't the database created before each test run?
<blake_r> jtv: its different than make syncdb
<jtv> Yes, but maybe the local database _cluster_ that lives inside the branch still had a remnant of UTC+1...
<jtv> (Wow, I don't know what type of animal I just slapped off my leg but it had a lot more rump than a mosquito and it spilled a lot of blood...)
<jtv> (Mine, of course!)
<blake_r> got a lot of mosquitos where your at?
<jtv> Ohhh yes.
<jtv> Not a lot of bites so far, I think.  These are not those little Euro bugs â you feel the sting WHILE they're biting you.
<blake_r> Damn.
<jtv> That's a funny change in habits I had to go through: when I felt a mosquito bite, I used to say to myself "oh well, it's already happened, all I can do now is try not to scratch it."  So I reacted slowly on purpose.
<jtv> Here, when you feel the bite, you need to hurry so you can still get rid of the bastard!
 * jtv looks accusingly at 3 geckos overhead that failed to catch this one
<blake_r> jtv: where are you? the jungle? lol
<jtv> Pretty much.
<onicrom> is there a way to make the fastpath installer the default?
<ctlaugh> I am trying to get some modifications to dhcpd.conf.template to take effect, but saving the cluster controller does not update /etc/maas/dhcpd.conf.
<ctlaugh> ^ Any suggestions?
<ctlaugh> (I am running trusty)
<jtv> onicrom: it's the default now.
<jtv> ctlaugh: try saving a cluster controller interface rather than just the cluster controller.  You may have to make a small change just to convince it.
<onicrom> jtv, not in the version thats installed with trusty
<jtv> onicrom: then marking a node for the fast-path installer sets a tag.
<ctlaugh> jtv: I tried that as well -- still not getting updated
<ctlaugh> ls -al
<onicrom> jtv is there a way to set that tag on all nodes upon creation?
 * ctlaugh typed in the wrong window by mistake
<jtv> onicrom: I think there is, but I haven't done it myself â set an xpath expression for the tag that simply always matches.
<jtv> onicrom: then for any new node, the system will see if it matches the xpath expression and if so, give it that tag.
<ctlaugh> jtv: Is there any way outside of the web ui (like a command line) to force an update?
<jtv> Thinking...
<jtv> You'd have to update a cluster interface field that actually affected the dhcp settings, such as increasing the dynamic IP range by 1 address (and then taking it back again if you care enough).
<onicrom> im beginning to think that the 'helpers' in curtin really dont do anything
<ctlaugh> jtv: That's what I did.  I changed on end of the address range.
<jtv> Hmmm... that ought to restart the dhcp server.
<jtv> Assuming it's an interface where maas manages dhcp.
<ctlaugh> jtv: Not only is it not picking up my changes (from the template), it's also not getting the range change from what I modified in the UI.
<ctlaugh> How can I verify that it thinks it's managing dhcp on that interface?
<ctlaugh> Something I also recently did was change the IP address of the maas server.
<jtv> Should be simple: the cluster must have been accepted into the maas, and its management setting should be either "DHCP" or "DHCP and DNS."
<ctlaugh> UI shows accepted, and interface shows "Manage DHCP and DNS"
<jtv> Owww, the mechanism that we use to carry messages from the region controller to the cluster controller, RabbitMQ, doesn't take too kindly to changing addresses.
<jtv> That would explain it, I think.
<jtv> (And yes, we're now in the process of getting rid of it!)
<ctlaugh> I ran dpkg-reconfigure on both maas-cluster-controller and maas-region-controller to fix the addresses, and think I rebooted after doing that (but possibly before)
<ctlaugh> Is there somewhere else that might need to be updated?
<jtv> Try a "grep -rI $OLD_ADDRESS /etc/rabbit*"
<ctlaugh> jtv: no matches, and /etc/rabbitmq is empty
<jtv> Oh, or /var/lib/rabbit* perhaps?
<jtv> We may store that stuff in the maas /etc or /var trees...  I remember running into this in the past and just grepping like mad.
<jtv> And it did turn up rabbit files.
<ctlaugh> jtv: ok, after a reboot and modifying the interface again, it worked.
<jtv> \o/
<ctlaugh> apparently, I changed the ip address in /etc/interfaces, rebooted, then ran dpkg-reconfigure
<ctlaugh> I should have restarted things after the reconfigure as well
<ctlaugh> jtv: thank you for your help
<jtv> Glad it worked out.
<jtv> As always, "have you tried turning it off and on again?"  :)
<ctlaugh> Yes :)
<dpb1> Hi -- has anyone noticed/reported that vms with virtio disks do not correctly register all their disks in MAAS?  Is that something that would be fixed if I reported a bug? :)
<onicrom> can someone running 1.5.2 verify what is on line 225
<onicrom> of /usr/lib/curtin/helpers/common
<blake_r> if I could get a review on both of these: https://code.launchpad.net/~blake-rouse/maas/add-bootresource-models/+merge/230161 and https://code.launchpad.net/~blake-rouse/maas/add-bootresource-models/+merge/230161
<blake_r> i was nice enough to split them up for the reviewer!
<blake_r> :)
<blake_r> crap same link twice, sorry: https://code.launchpad.net/~blake-rouse/maas/bootresource-model-properties/+merge/230164
<jtv> blake_r: having a look, and liking the documentation so far...
<jtv> blake_r: reviewed one.
<onicrom> can someone point me to the scripts/source around cloud-init ?
<onicrom> it's doing something to the partition table... i ran a set -x on the curtin partition scripts and nother in there is doing this
<onicrom> can someone point me to the code that does the initial boot after the 'start' is completed
<thewmf> howdy maas people
<thewmf> I'm trying to get multiple NICs to work with curtin but the blog post doesn't make sense to me
<thewmf> anybody interested?
<onicrom> im having a fun time with curtin :)
<thewmf> so you're into S&M?
<onicrom> i wanna poke out my eyeballs
<thewmf> ok, that's the same as my experience
<thewmf> I actually have experience with d-i, but I don't think it's working
<onicrom> im trying to change the default partition size
<onicrom> its nearly impossible and im hacking code
<thewmf> I saw that thread on the list. not promising
<thewmf> I hate the d-i partitioner
<newell> onicrom, still hitting your head against the way with curtin?
<newell> ;)
<onicrom> dood
<onicrom> im dying here
<onicrom> im about to change the grub to drop me to a shell
<newell> onicrom, the only code that I have modified for curtin so far has been to add the ability to add a separate boot partition, which is only supported for gpt currently
<onicrom> and howd that work?
<onicrom> i switched to gpt
<newell> good, its in trunk now
<newell> what you will need to do is you will need to make new block_meta commands
<newell> to be able to set the size
<onicrom> can you paste me like to the code?
<newell> take a look at curtin/curtin/commands/block_meta.py
<onicrom> also.... the part that is confusing me is im able to get it to create the partition  layout i want
<newell> this is the code that calls the "partition" script which then calls the "common" script
<onicrom> and when helpers/common runs and exits sgdisk -p /dev/sda
<onicrom> prints out what i want
<onicrom> then it reboots and its gone and changed sda1 to be the full isize of the disk
<newell> are you modifying templates?
<onicrom> where are said templates?
<newell> let me re-phrase my question
<newell> How are you calling curtin?
<newell> or are you just hacking the code in curtin?
<onicrom> hacking the code in curtin
<onicrom> well ... im editing the common shell script which is exected by the partition shell script which is executed by block_meta
<newell> okay so yeah you can go about it that way too
<onicrom> have you seen my long ass bug?
<newell> no, do you have a link?
<onicrom> https://bugs.launchpad.net/maas/+bug/1354426
<ubot5> Ubuntu bug 1354426 in MAAS "curtin creates a 2TB root fs/partition by default" [Undecided,New]
<onicrom> the most relevant bit where i have it outputting exactly what im doing
<onicrom> then the install finishes and the damn gremlins did somethin
 * newell takes a look
<newell> ha...the gremlins
<newell> is there any way you can convert to the newest code?
<onicrom> sure might need help :)
<newell> okay no problem
<newell> I can help
<newell> $ sudo apt-get install bzr
<newell> you need bzr to check out the source
<onicrom> ok
<newell> once you have bzr you can create a branch
<onicrom> give me 2 mins
<onicrom> need to potty break
<newell> $ bzr branch lp:curtin curtin
<newell> that will make a directory called curtin for you
<newell> $ cd curtin
<newell> $ ./tools/build-deb -uc -us
<newell> The above command will build all the *.deb packages for you that you need
<newell> then you can install them doing:
<newell> $ sudo dpkg -i *.deb
<onicrom> done
<newell> onicrom, everything worked?
<newell> for install that is
<onicrom> yes im going to rebuild one box
<onicrom> just to see
<newell> onicrom, so now you know how to take the source and make a package out of it
<newell> so, you can hack it up, then install it etc
<onicrom> thats pretty sweet
<newell> the results you were mentioning were for hardware correct?
<onicrom> you mentioned etemplates
<newell> not the devel stuff?
<onicrom> what do you mean hardware results/
<newell> I mentioned to you in the email how if you follow doc/devel/README.txt it will show you how to develop for curtin with qemu images
<newell> I was wondering if the results you mentioned in the bug report are for that (qemu) or on actual hardware?
<onicrom> ohhhh that yes hardware
<onicrom> everything im doing is on baremetal
<newell> k
<onicrom> i have 20 servers im trying to get ready for openstack :)
<newell> nice :)
<newell> so let me know what issues you run into later on but at least you have the most current code installed
<onicrom> i only have ipmi access via serial over lan... and the debian installer hoses the network for ipmi somehow (the nic is shared between the ipmi and the os)
<onicrom> so im forced to use curtin
<onicrom> as it does not do such network fuckery
<onicrom> do you know what happens with regards to partitions/disks upon first boot after the curtin bit is run?
<newell> you mean after curtin installs?
<onicrom> correct
<onicrom> ok so that didnt work
<onicrom> i have no /boot
<newell> you need to use the simple-boot command
<newell> not the default simple
<newell> currently it is dependent on which architecture you have
<onicrom> how do i select
<onicrom> change in block_meta or ?
<newell> onicrom, give me a second
<onicrom> for you
<onicrom> take 2!
<newell> onicrom, sorry about that
<onicrom> np i hard coded
<newell> take a look at doc/topics/overview.rst
<onicrom> boot=true
<newell> yeah you can do that too ;)
<newell> that will make you a 512MiB separate /boot partition on /dev/sda1
<newell> and the rest of the disk will have / partition on /dev/sda2
<onicrom> i changed that to +50G
<onicrom> i hope hope hope that works
<onicrom> $end is bad for me :)
<newell> yeah curtin could really use this feature
<onicrom> ok good news
<onicrom> no boot partition
<onicrom> BUT
<onicrom> root is 50gG!@!!!!!!@#$@%$@#%
<newell> how are you verifying the partitions?
<onicrom> sorry let me rephrase
<onicrom>  /boot is there, but there is no fstab reference
<newell> that is because you hardcoded it
<newell> the fstab code is in block_meta.py
<onicrom> ah :)
<newell> yeah its a cluster fsck
<newell> onicrom, what you can do is you can hard code the default to get the fstab
<newell> instead of doing the hack in common
<newell> change...
 * newell looks where it is
<newell> install.py at the top has block-meta using 'simple'
<newell> change that to 'simple-boot'
<newell> that or you can specify the partitioning command as the overview.rst shows
<onicrom> lemme try that one
<onicrom> i still need to hack common
<newell> yeah for the size part you will
<onicrom>             --new "1::+1G" --typecode=1:8300 \
<onicrom>             --new "2::+50G" --typecode=2:8300 "$target" ||
<onicrom> it be nice to pass those variables just lke passing simple-boot
<newell> file a bug report for the ability to create custom boot partitions (i.e. the size)
<onicrom> hell it would be simple to pass 1:size:mount and have common iterate over an array
<onicrom> as well as block_meta
<newell> yeah shouldn't be too hard
<newell> feel free to add it and contribute if you like :)
<onicrom> i would put some disclaimer... if you put in numbers here that do not match the disks available its your own damn fault
<onicrom> i mean right now it tries to be smart about the sizes and what not but i say let people shoot themselves in the foot :)
<onicrom> would paritioning_commands:./curtin/commands/block_meta.py:               'choices': ['raid0', SIMPLE, SIMPLE_BOOT]}),
<onicrom> inside curtin_preseed?
<newell> onicrom, did you take a look at overview.rst?
<newell> it shows how to do the partitioning commands
<onicrom> vi ues
<onicrom>   50_setup_raid: curtin disk-setup --all-disks raid0 /
<newell> yes
<newell> so that gives examples
<onicrom> change raid0 to SIMPLE_BOOT
<newell> no
<newell> you would do similar to what is install here
<newell> 00_something_blah: curtin block-meta simple-boot...
<newell> something along those lines
<newell> similar to install.py*
<onicrom> ahhh block_meta
<newell> I have never done it this way so not sure on the exact syntax
<newell> but I know install.py is doing this (those are the defaults at the top if nothing in the configuration sets it)
<onicrom>     'partitioning_commands': {
<onicrom>         'builtin': ['curtin', 'block-meta', 'simple']},
<onicrom> there we go
<onicrom> paritioning_commands: 10_doshit: ['curtin', 'block-meta', 'simple-boot']
<onicrom> i really need to pick up python :)
<newell> yeap
<newell> so where you going to put that last line?
<onicrom>  /etc/maas/preseeds/curtin_userdata
<newell> yeap ;)
<onicrom> i threw it after early_commands and before late_commands
<newell> go ahead and give that a go and let me know how it works
<newell> correct
<newell> that is what overview.rst says at least
<onicrom> its running now :)
<onicrom> ok cross your fingers the install is going
<onicrom> the part that was confusing me earlier... i did a set -x in the common script saw exactly what was being run
<onicrom> and i was at a complete loss as to how the partition size was changing
<newell> cool, let me know how it goes
<onicrom> question about python  with the file.py and file.pyc
<onicrom> will it exec the pyc? so the contents of the py could have been one way and the pyc did something compeltely different?
<newell> pyc is executed by if there are any modifications to the original py before running it will re-compile the byte code for the pyc
<onicrom> ok.
<newell> so you should be good, as long as you saved the file ;)
<onicrom> also my new machine now has a 2.7TB fs ....
<newell> is that what you wanted?
<onicrom> it juuust had a 50G wtf
<onicrom> nope
<onicrom> common has the code changes the only difference this time was boot=true was set to false
<newell> you changed the curtin_userdata?
<onicrom> correct
<newell> did it at least create the separate /boot partition?
<onicrom> but i would think even without that change it would have executed common which would have done:  --new "1::+50G" --typecode=1:8300 "$target" ||
<newell> what does sudo mount say when you login to the node
<newell> ?
<newell> what common file are you editing?
<newell> did you re-create the package again?
<newell> also for maas, when you change the curtin_userdata...which file are you changing?
<newell> all those things matter
<onicrom> mount
<onicrom>  /dev/sda1 on / type ext4 (rw)
<onicrom> and the normal tmpfs stuff
<newell> k, what about the other questions?
<onicrom>  /etc/maas/preseeds/curtin_userdata
<onicrom> editing: /usr/lib/curtin/helpers/common
<newell> maybe print to the console to make sure you are executing what you think you should be in common?
<onicrom> im going to set -x that bitch!
<newell> :)
<newell> I have never used that, how does it work?
<onicrom> i was very close to adding a /bin/bash to the grub line it enters to drop me to a prompt before the first boot
<onicrom> lol
<onicrom> set -x in bash is essentially debug mode, it will show you everything its doing
<newell> cool never used it but playing with it now
<newell> so why are you setting up different partitions, is that something you think you would use often?
<onicrom> i want to use lvm for most of my storage so i can do lvm snapshots for my lxc containers
<onicrom> if i also want to setup drdb replication
<onicrom> linux has always has /boot as a non-journal'd filesystem for safety
<onicrom> so i want that and generally i throw everything in lvm volumes so that if i need to grow them later its super easy
<onicrom> ok so something broke :)
<newell> what did set -x tell you?
<onicrom> https://paste.tinyw.in/index.php/view/11650792
<onicrom> it didnt get that far this time..
#maas 2014-08-09
<newell> you hacked something so there is only simple-boot
<newell> looks like you changed install.py
<newell> change it back to what it originally was
<onicrom> i changed block_meta.py
<newell> okay change whatever python you changed back to the original
<newell> now that you are using the partitioning_command you shouldn't need those hacks
<onicrom> done
<newell> it also looks like it is not running what you put in curtin_userdata for some reason
<onicrom> it did because it hit what i told it to run
<newell> but go ahead and give it another go
<newell> okay
<onicrom> does it need to be in that weird looking array
<onicrom> or can i just run commands in there?
<onicrom> 10_doshit: ['curtin', 'block-meta', 'simple-boot']
<onicrom> that thing
<newell> paste your file
<onicrom> https://paste.tinyw.in/index.php/view/84388124
<onicrom> ok this is gonna blow your mind...
<onicrom> https://paste.tinyw.in/index.php/view/5593153
<onicrom> so the simple-boot bit didnt work... BUT my 50G partition did ger created
<onicrom> im willing to bet though that when i login for the first time, im going to see 1 2.7T root filesystem
<onicrom> yup!
<onicrom> https://paste.tinyw.in/index.php/view/63284151
<onicrom> its like whatever was done in that curtin session did not *actually* write the partition table
<newell> so the simple-boot part...is architecture dependent
<newell> I forgot that part
<onicrom> <x86_64
<onicrom> aka amd64
<newell> yeah it only happens for arm64
<newell> stupid I know, but that is how it is setup currently
<newell> so that is why it looks like that wasn't run
<onicrom> well im using amd64
<onicrom> amd64 == x86_64
<newell> amd64 != arm64
<onicrom> ohhhh arm64!
<newell> ;)
<onicrom> sorry
<newell> no worries
<onicrom> see my other pastes?
<newell> yeah give me a couple minutes...trying to eat dinner with the wife
<newell> I think you are on the right track
<newell> basically what is happening is simple is being run...but why you are not getting the partition you think...I will have to look more at the pastes
<newell> onicrom, you sure you are booting into the correct node?
<newell> cause yeah....I am scratching my head
<newell> logging in I mean
<onicrom> ive been looking at these guys so much
<onicrom> its possible
<onicrom> i will confirm in a oment
<onicrom> i was dinner, i had burgers :)
<onicrom> ci-info: |  eth0  |  True | 10.211.25.216 | 255.255.255.0 | 78:45:c4:f4:65:56 |
<onicrom> ok its working again but i added some more stuff
<newell> so what was the issue?
<onicrom>     sgdisk -Z "$target"
<onicrom>     sgdisk -og "$target"
<onicrom>     sgdisk -p "$target"
<onicrom> then lket the hardcoded sizes go
<onicrom> and run     sgdisk -v "$target"
<onicrom> -Z blow away, -og recreate
<onicrom> -v verify
<onicrom> lket=let
<onicrom> i also added:
<onicrom> paritioning_commands: 10_wipeshit: curtin wipe --quick --all-unused-disks
<newell> I think your wipeshit was what probably helped
<onicrom> no idea what the issue was but i thought id blow it out of the water wiping the partition(s)/table
<onicrom> where is defined that only arm64 will work btw?
<newell> block_meta.py
<newell> machine.platform logic
<onicrom> is that as simple as changing it?
<newell> yeap :)
<onicrom> 'amd64' instead of aarch64
<newell> yeah if you do that it will give you a separate boot partition
<newell> make sure to check for other instances of aarch64 though
<onicrom> i only found that in all of curtina
<onicrom> and the hooks
<onicrom> ./commands/curthooks.py:    if machine.startswith('armv7') or machine.startswith('aarch64'):
<onicrom> ok sir fingers crossed!
<onicrom> do you know if /boot is meant to be the first parition
<onicrom> because it looks like whatever just happened it went to /dev/sda1 for /
<onicrom> and ran out of space :)
<onicrom> dont think the simple-boot is being done
<onicrom> nope
<onicrom> echo "i am running $0 with $@"
<onicrom> i am running /curtin/helpers/partition with /dev/sda
<onicrom> i am running /curtin/helpers/install-grub with /tmp/tmpYXa48u/target /dev/sda
<onicrom> ok that is interesting
<onicrom> the only thing i just changed was boot=true to boot=false inside common
<onicrom> and now my / is one giant 2.7T filesystem again
<newell> onicrom, you are doing something! lol
<newell> I know simple boot works as is for arm64
<newell> you need to open a python terminal on the node architecture if possible
<onicrom> sorry whats that mean?
<newell> I suspect that your architecture, at least from python is not what you expect
<newell> so do this
<newell> login into one of your nodes
<newell> $ python
<newell> >>> import platform
<newell> >>> print platform.machine()
<newell> and see what it tells you
<onicrom> x86_64
<onicrom> let me execute on the server itself
<onicrom> same thing
<onicrom> oh
<onicrom> im an idiot me thinks
<onicrom> *facepalm*
<onicrom> i put amd64
<onicrom> haha
<newell> ;)
<onicrom> ive been screwing with this since yeterday
<onicrom> forgive my bug eyes
<newell> no worries, I understand
<newell> I have to run
<newell> email me when you get it working...cause you are super close
<onicrom> you have been a very big help
<onicrom> thank you so much
<newell> no problem
<jason___> anybody have any luck with maas on an HP c7000?
<jason___> is there a channel for setup help?
#maas 2014-08-10
<jason___> one or more mac addresses invalid?
#maas 2015-08-03
<mup> Bug #1044421 changed: Password field for IPMI configuration is clear text <MAAS:Fix Released> <https://launchpad.net/bugs/1044421>
<mup> Bug #1461712 changed: 1.8rc1:  sm15k systems - non-PXE NIC's mac address doesn't get linked back to cluster <oil> <MAAS:Expired> <https://launchpad.net/bugs/1461712>
<mup> Bug #1461712 opened: 1.8rc1:  sm15k systems - non-PXE NIC's mac address doesn't get linked back to cluster <oil> <MAAS:Expired> <https://launchpad.net/bugs/1461712>
<mup> Bug #1461712 changed: 1.8rc1:  sm15k systems - non-PXE NIC's mac address doesn't get linked back to cluster <oil> <MAAS:Expired> <https://launchpad.net/bugs/1461712>
<tai271828> hi, is it possible to assign FQDN domain for nodes?  .maas may be too generic and I want to have my own one for lab infrastructure.
<roaksoax> tai271828: you can edi tyour cluster controller and change that
<tai271828> roaksoax, cool, thanks!
<tai271828> roaksoax, let me try.
<mup> Bug #1480996 opened: maas memory rounding seems wrong <maas (Ubuntu):New> <https://launchpad.net/bugs/1480996>
<mup> Bug #1480996 opened: maas memory rounding seems wrong <MAAS:New> <maas (Ubuntu):New> <https://launchpad.net/bugs/1480996>
<geekinutah> Looking for maas packages that work, the installation guide suggestst that I add the cloud-archive:tools PPA
<geekinutah> when I do so I get this error -> http://pastebin.com/mNtMGt1G
<catbus1> geekinutah: are you running 12.04 or 14.04?
<geekinutah> 14.04
<geekinutah> catbus1: ^
<catbus1> geekinutah: that should work, let me create a 14.04 real quick
<catbus1> I get the same error on 14.04.
<catbus1> jamespage: ^^^ File "/usr/lib/python3/dist-packages/softwareproperties/cloudarchive.py", line 99, in expandÂ  Â  if codename not in (MAP[self.caname]['release'],KeyError: 'release'
<jamespage> cloud-tools was only for 12.04 catbus1
<jamespage> but that is a crappy error message
<jamespage> the maas team run a stable ppa - I'd defer to them for location
<catbus1> geekinutah: ^^^ the stable ppa for maas is ppa:maas-maintainers/stable, that will get you maas 1.8, that's newer than 1.7.6 in the trusty-updates/main.
<geekinutah> catbus1, jamespage: thanks
<geekinutah> catbus1, jamespage: Where do I file the documentation bug? I got the wrong directions from here -> http://maas.ubuntu.com/docs/install.html#installing-maas-from-the-archive
<catbus1> geekinutah: https://bugs.launchpad.net/maas
<geekinutah> right, duh, thx
<mup> Bug #1481091 opened: MaaS Install guide incorrectly refers to 12.04 cloud PPA <MAAS:New> <https://launchpad.net/bugs/1481091>
<mup> Bug #1481091 changed: MaaS Install guide incorrectly refers to 12.04 cloud PPA <MAAS:New> <https://launchpad.net/bugs/1481091>
<mup> Bug #1481091 opened: MaaS Install guide incorrectly refers to 12.04 cloud PPA <MAAS:New> <https://launchpad.net/bugs/1481091>
#maas 2015-08-04
<cjohnston> Running MAAS 1.7.6, I'm doing "maas login myprofile ....", which says "You are now logged in...."... After that, I run "maas myprofile node-groups import-boot-images" and get the feedback "Authorization Error: Invalid API key."  - any ideas why I'd be getting an Invalid API key error immediately after successfully logging in?
<mup> Bug #1481118 opened: maas-region-admin apikey command missing --username <MAAS:New> <https://launchpad.net/bugs/1481118>
<robertchen> hello, I am trying to install Centos 6 on a MAAS cluster. The steps I was taken as follows, discover the nodes -> commission the nodes -> acquired the nodes -> edit the node with the "Architecture: amd64/generic; OS: Centos; release: Centos 6.0" -> start the nodes, during which I had to power up the nodes with the push button because the nodes cannot power on automatically. I got the install said installation finished, but the staut
<robertchen> failed. Can someone help me to solve this problem?
<mup> Bug #1481261 opened: MAC address text field on Power section accepts wrong format <MAAS:Incomplete> <https://launchpad.net/bugs/1481261>
<mup> Bug #1481275 opened: 1.8 - User should be able to have more than one accordion panel open <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1481275>
<mup> Bug #1481276 opened: 1.8 Hit box for deleting a tag is small <ui> <MAAS:New> <https://launchpad.net/bugs/1481276>
<mup> Bug #1481277 opened: 1.8 Can't tag node "HP" <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1481277>
<mup> Bug #1481275 changed: 1.8 - User should be able to have more than one accordion panel open <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1481275>
<mup> Bug #1481276 changed: 1.8 Hit box for deleting a tag is small <ui> <MAAS:New> <https://launchpad.net/bugs/1481276>
<mup> Bug #1481277 changed: 1.8 Can't tag node "HP" <ui> <ux> <MAAS:New> <https://launchpad.net/bugs/1481277>
<mup> Bug # opened: 1481275, 1481276, 1481277, 1481281, 1481283, 1481285
<rick_h_> ty all, upgraded the team maas to 1.8 and added another machine and <3 https://plus.google.com/+RickHarding/posts/a1Ziofr7FLx
<rick_h_> don't tell anyone, but my favorite cloud is maas :P
#maas 2015-08-05
<vibol> Don't know where to boot the image in maas lastest version!
<vibol> Anyone here ?
<catbus1> vibol: what do you mean?
<vibol> the old version there and botton on the gui to import boot image
<vibol> but the new gui don't see where to import the boot image... let me screenshot it for u
<catbus1> vibol: Do you see the Images tab on the top? The third one from the left
<vibol> yeb i got to http://ip/MAAS/image
<vibol> http://postimg.org/image/4javz73h5/ catbus1
<catbus1> it says there is no boot sources to download the images from
<catbus1> let me see what the official doc says about creating boot source
<vibol> Thank for help @catbus1
<catbus1> vibol: that's odd, a default boot source should come with the maas install.
<vibol> yes! this is i already espected
<vibol> i'm really don't about this
<vibol> don't understand*
<catbus1> vibol: I have the same 1.8 version, but it was an upgrade from 1.7. One thing we can do here is to manually add the source via CLI.
<catbus1> https://maas.ubuntu.com/docs/bootsources.html#adding-a-source
<vibol> so i don't see where is the configure file located let me see!!
<vibol> That strenge @catbus1
<vibol> the boot image source is already configure the the correct url http://postimg.org/image/j4i5o96n3/
<catbus1> it is configured.
<catbus1> maybe it's something between clusterd and regiond
<catbus1> I'd look at /var/log/maas/clusterd.log and /var/log/maas/regiond.log to see if there is any error messages about boot sources
<catbus1> dinner
<vibol> hmm.. nothing wrong on the log
<vibol> ..........
<mup> Bug #1439476 changed: Editing a cluster interface can cause an internal server error <oil> <MAAS:Expired> <https://launchpad.net/bugs/1439476>
<mup> Bug #1457585 changed: 1.8b7: Empty list of all nodes and incorrect number of total nodes shown in UI <oil> <MAAS:Expired> <https://launchpad.net/bugs/1457585>
<mup> Bug #1481759 opened: boot-images reports images, but pxe boot fails with no-such-image <landscape> <MAAS:New> <https://launchpad.net/bugs/1481759>
<mup> Bug #1481759 changed: boot-images reports images, but pxe boot fails with no-such-image <landscape> <MAAS:New> <https://launchpad.net/bugs/1481759>
<mup> Bug #1481759 opened: boot-images reports images, but pxe boot fails with no-such-image <landscape> <MAAS:New> <https://launchpad.net/bugs/1481759>
<mup> Bug #1481813 opened: Failed trusty deployment on Lenovo X3650 M5 in UEFI mode <oil> <curtin:New> <MAAS:New> <https://launchpad.net/bugs/1481813>
<mup> Bug #1481874 opened: New node cannot enlist due to internal server error <MAAS:New> <https://launchpad.net/bugs/1481874>
<mup> Bug #1384453 changed: Feature: it would be nice if the summary page (the pie-chart page) showed how many total cores, ram and GB of storage we have. <MAAS:Fix Released> <https://launchpad.net/bugs/1384453>
<mup> Bug #1386923 changed: It would be nice to have an easier way to see a node's status (ready vs deployed vs whatever) <confusing-ui> <MAAS:Fix Released> <https://launchpad.net/bugs/1386923>
<mup> Bug #1389103 changed: Unable to install TD350 Server in EFI mode using MAAS <server-hwe> <MAAS:Invalid> <https://launchpad.net/bugs/1389103>
<mup> Bug #1481940 opened: Failure in MAAS startup messages <MAAS:New> <https://launchpad.net/bugs/1481940>
<mup> Bug #1481940 changed: Failure in MAAS startup messages <MAAS:New> <https://launchpad.net/bugs/1481940>
<mup> Bug #1481940 opened: Failure in MAAS startup messages <MAAS:New> <https://launchpad.net/bugs/1481940>
<mup> Bug #1481970 opened: VM stuck in comissioning - 500 Server error /metadata/2012-03-01 <MAAS:New> <https://launchpad.net/bugs/1481970>
#maas 2015-08-06
<mup> Bug #1481992 opened: Upgrade of grub-pc during install fails <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1481992>
<mup> Bug #1397952 changed: DNS breaks for multihomed host <canonical-bootstack> <canonical-is> <MAAS:Expired> <https://launchpad.net/bugs/1397952>
<jogarret6204> Hi folks. i'm having trouble getting maas 1.8 to work with 1404.  does this combination work, or should I back down to something in 1.7?
<jogarret6204> can't provision VMs.  can't bootstrap juju
<jogarret6204> https://bugs.launchpad.net/maas/+bug/1481970
<roaksoax> jogarret6204: is maas-dhcpd running?
<roaksoax> jogarret6204: can you test sudo add-apt-repository ppa:maas-maintainers/experimental
<roaksoax> jogarret6204: and then sudo apt-get update && sudo apt-get dist-upgrade -> to upgrade MAAS
<jogarret6204> trying now
<roaksoax> jogarret6204: this is weirD:
<roaksoax>  provisioningserver.service_monitor.UnknownServiceError: 'maas-dhcpd' is unknown to upstart.
<roaksoax> if it is maas on trusty, i wonder why that would be unknown to upstart
<jogarret6204> saw soemthing in release notes about transitioning to systemd...  was wondering if I was caught in transition phase
<jogarret6204> ..by using 1.8 on trusty..
<roaksoax> jogarret6204: can you show me your sudoers ?
<roaksoax> jogarret6204: /etc/sudoers.d/99-maas-sudoers
<roaksoax>  ?
<jogarret6204> maas ALL= NOPASSWD: /usr/sbin/service maas-dhcpd start
<jogarret6204> its about 20 lines - where should I post it?
<roaksoax> jogarret6204: pastebin ?
<roaksoax> jogarret6204: http://paste.ubuntu.com/12013755/
<roaksoax> jogarret6204: does it look like that?
<jogarret6204> yes - identical...  interesting that all three of these show up in logs asn not recognized by upstart
<jogarret6204> Aug  6 09:42:54 maas maas.service_monitor: [ERROR] While monitoring service 'tgt' an error was encountered: 'tgt' is unknown to upstart.
<roaksoax> jogarret6204: are you running on trusty with systemd?
<roaksoax> jogarret6204: by installing 1.8.1 no fix either?
<jogarret6204> idk how can I tell?  I'm a network guy.  just learning linux
<roaksoax> jogarret6204: sudo service maas-dhcpd status ?
<roaksoax> ls /var/lib/systemd ?
<roaksoax> blake_r_: https://bugs.launchpad.net/maas/+bug/1481970
<jogarret6204> oh - I know that service is running, yes
<jogarret6204> now sure how to tell if I am running systemd
<roaksoax> jogarret6204: ls /var/lib/systemd ?
<jogarret6204> think I am running it
<jogarret6204> http://pastebin.com/m6SRBwAu
<roaksoax> jogarret6204: nope you are not
<jogarret6204> no?  what is the indicator?
<roaksoax> jogarret6204: /lib/systemd/systemd
<blake_r_> jogarret6204: does /etc/init/maas-dhcpd.conf exist?
<jogarret6204> will check - restarting now.  assume it is because I provisioned 10 bare-metal machinse yesterday.. oever with the failures today, mahine gets IP, then gives it back before failing
<jogarret6204> yes.  exists.  interesting that you ask, because I rebuilt this entire machine yesterday because maas-dhcp process was not starting, /maas/dhcpd.copnf fiel was not getting created was in the logs
<jogarret6204> but on this rebuild the maas-dhcp is starting ok
<jogarret6204> same fail as the bug with version 1.8
<jogarret6204> UnhandledCommand: ('UNHANDLED', 'Unknown Error [maas:pid=973:cmd=RemoveHostMaps:ask=18]')
<jogarret6204> maas-dhcpd.conf file does exist
<blake_r_> jogarret6204: can you do "sudo service maas-dhcpd status"
<blake_r_> jogarret6204: and provide the output
<jogarret6204> maas-dhcpd start/running, process 1656
<mup> Bug #1432666 changed: [FFe] New upstream release 1.8.0 <upgrade-software-version> <maas (Ubuntu):Fix Released> <https://launchpad.net/bugs/1432666>
<blake_r_> jogarret6204: wierd
<jogarret6204> added the new clusterd trace tothe bug, but line numbers were the same
<blake_r_> jogarret6204: ah check your sudoers
<blake_r_> jogarret6204: /etc/sudoers.d/maas
<blake_r_> jogarret6204: what is the content of that?
<jogarret6204> is is same as roaksoax had posted earlier
<jogarret6204> http://paste.ubuntu.com/12013755/
<jogarret6204> one thing my be dofferent on my system
<jogarret6204> I have libvirt and using kvm on this machine to make my VM for juju to use for deployment
<jogarret6204> so I added a maas user myself, wiht a password.
<jogarret6204> would that conflict at all here?  since I see this maas user being used a lot
<jogarret6204> was adding my maas user to libvirt for power control of the VM
<jogarret6204> @blake_r_ - any other ideas?  or should I back down into 1.7? I need to get this environment up today. I do appreciate the help from both you and roalksoax
<blake_r_> jogarret6204: is the maas user part of the maas group?
<jogarret6204> i was alloweed to add it so I assume not.
<jogarret6204> retrying a provision after adding  - no dice.,  same exceptions and fail
<blake_r_> jogarret6204: you will need to restart the system
<jogarret6204> k - restarting
<jogarret6204> failed again.  also GUI for nodes hangs with "connecting" status.  restarting one more time
<jogarret6204> other gui screens avaialable at the time..  just the MAAS/#/nodes hangs at "connecting"
<jogarret6204> sorry - browser issue.  works fin in chrime.  IE is "connecting"
<blake_r_> jogarret6204: IE does not support websockets
<blake_r_> jogarret6204: so that didn't work?
<jogarret6204> nope.  just tried again
<blake_r_> jogarret6204: you have something not configured correctly since you created the maas user manually
<blake_r_> jogarret6204: the maas user needs to be able to use sudo for those commands
<blake_r_> jogarret6204: try this "sudo -u maas -H server maas-dhcpd status"
<blake_r_> jogarret6204: if that fails then your maas user is incorrect
<jogarret6204> that works
<jogarret6204> but I do see issues in syslog
<jogarret6204> maas sudo: pam_unix(sudo:auth): auth could not identify password for [maas]
<blake_r_> jogarret6204: maas should require no password to execute that command
<blake_r_> jogarret6204: something is incorrect with your maas user and sudoers
<jogarret6204> I think you got it!  changed visudo from:  maas ALL=(ALL)  ALL  to maas ALL=(ALL) NOPASSWD: ALL
<jogarret6204> juju is bootstrapping
<blake_r_> jogarret6204: awesome!
<blake_r_> jogarret6204: glad you figured it out
<blake_r_> jogarret6204: let me know if you have any more questions or run into any more issues
<jogarret6204> yep.  all me.  :-)  - Your helpis much appreciated!
<jogarret6204> roaksoax thanks to you as well.
<roaksoax> jogarret6204: was this a clean systme?
<roaksoax> or what changed that?
<blake_r_> roaksoax: no he created the maas user manually
<blake_r_> roaksoax: it was part of the maas user group, and he made changes manually
<roaksoax> blake_r_: gotcha!
<jogarret6204> it was a clean build, then updated.  Added libvirt, added maas user when adding libvirt for power control
<jogarret6204> some internal notes I had using maas as the virsh user...
<mup> Bug #1481970 changed: 'maas-dhcpd' is unknown to upstart. <MAAS:Invalid> <https://launchpad.net/bugs/1481970>
<mup> Bug #1455770 changed: 100 GB disk shows as 1 in GUI <apport-collected> <sts> <tests> <third-party-packages> <trusty> <MAAS:Fix Released> <MAAS 1.7:Won't Fix> <https://launchpad.net/bugs/1455770>
<mup> Bug #1482385 opened: boot-sources disappear after import <cisco> <landscape> <MAAS:New> <https://launchpad.net/bugs/1482385>
<mup> Bug #1482385 changed: boot-sources disappear after import <cisco> <landscape> <MAAS:New> <https://launchpad.net/bugs/1482385>
<mup> Bug #1482385 opened: boot-sources disappear after import <cisco> <landscape> <MAAS:New> <https://launchpad.net/bugs/1482385>
<mup> Bug #1482405 opened: Unhandled failure dispatching AMPQ command <oil> <MAAS:New> <https://launchpad.net/bugs/1482405>
<mup> Bug #1482405 changed: Unhandled failure dispatching AMPQ command <oil> <MAAS:New> <https://launchpad.net/bugs/1482405>
<mup> Bug #1482405 opened: Unhandled failure dispatching AMPQ command <oil> <MAAS:New> <https://launchpad.net/bugs/1482405>
#maas 2015-08-07
<mup> Bug #1482441 opened: maas-cli network connect-macs succeeds with invalid parameters when it should fail <cisco> <landscape> <MAAS:New> <https://launchpad.net/bugs/1482441>
<mup> Bug #1482563 opened: validate_in_transaction() error message is too specific <MAAS:Triaged> <https://launchpad.net/bugs/1482563>
<mup> Bug #1482589 opened: test_size_is_rounded_to_next_block is flaky <MAAS:New> <https://launchpad.net/bugs/1482589>
<dooferlad> hi all, I have a MAAS here that has run out of addresses in the static pool and I want to delete all the existing allocations. Can someone help with that?
<dooferlad> I guess POST op=release with the appropriate params to /MAAS/api/1.0/ipaddresses/ repeatedly. I feel a script coming on.
<dooferlad> https://gist.github.com/dooferlad/082996faeb2168a76ab4 did it
<mup> Bug #1482737 opened: Cannot acquire a node AttributeError: This QueryDict instance is immutable <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1482737>
<mup> Bug #1482778 opened: Sorting by ram shows 3-digit machines at bottom of list <sts> <MAAS:New> <https://launchpad.net/bugs/1482778>
<mup> Bug #1482784 opened: When filtering nodes by hostname in the API, have to use ".local" domain instead of the domain set for the cluster <oil> <MAAS:New> <https://launchpad.net/bugs/1482784>
<mup> Bug #1482784 changed: When filtering nodes by hostname in the API, have to use ".local" domain instead of the domain set for the cluster <oil> <MAAS:New> <https://launchpad.net/bugs/1482784>
<mup> Bug #1482784 opened: When filtering nodes by hostname in the API, have to use ".local" domain instead of the domain set for the cluster <oil> <MAAS:New> <https://launchpad.net/bugs/1482784>
#maas 2015-08-08
<mup> Bug #1482853 opened: formencode.api.Invalid: That is not a valid URL <MAAS:New> <https://launchpad.net/bugs/1482853>
<mup> Bug #1482853 changed: formencode.api.Invalid: That is not a valid URL <MAAS:Invalid> <https://launchpad.net/bugs/1482853>
<mup> Bug #1482857 opened: Move dhcpd6.conf and dhcpd.conf to /var/lib/maas  <MAAS:Confirmed for andreserl> <https://launchpad.net/bugs/1482857>
<mup> Bug #1482858 opened: Move maas-http.conf to /var/lib/maas <MAAS:Triaged by andreserl> <https://launchpad.net/bugs/1482858>
<mup> Bug #1482859 opened: Move maas-proxy.conf to /var/lib/maas <MAAS:Confirmed for andreserl> <https://launchpad.net/bugs/1482859>
#maas 2015-08-09
<DrewT> I just upgraded to 1.8.0+bzr4001-0ubuntu2 and noticed that the debian-installer is no longer an option in the node config on the gui or cli, is this a bug or is that feature now deprecated?
#maas 2016-08-08
<mup> Bug # changed: 1518414, 1533855, 1540766, 1544143, 1561259, 1590406
<mup> Bug #1610954 opened: Misleading steps in http://maas.ubuntu.com/docs2.0/install.html#installing-maas-in-a-lxc-container <MAAS:New> <https://launchpad.net/bugs/1610954>
<mup> Bug #1610955 opened: Typo in http://maas.ubuntu.com/docs2.0/install.html#post-install-tasks <MAAS:New> <https://launchpad.net/bugs/1610955>
<mup> Bug #1610955 changed: Typo in http://maas.ubuntu.com/docs2.0/install.html#post-install-tasks <MAAS:Invalid> <https://launchpad.net/bugs/1610955>
<mup> Bug #1610955 opened: Typo in http://maas.ubuntu.com/docs2.0/install.html#post-install-tasks <MAAS:Invalid> <https://launchpad.net/bugs/1610955>
<mup> Bug #1610955 changed: Typo in http://maas.ubuntu.com/docs2.0/install.html#post-install-tasks <MAAS:Invalid> <https://launchpad.net/bugs/1610955>
<Sina_Maleki> Hi all
<Sina_Maleki> I have 8 hp server for testing openstack via maas and landscape
<Sina_Maleki> So what cisco switch model do you offer ?
<Sina_Maleki> Anyone there ?
<tatric> hello
<tatric> anyone using maas 2?
<roaksoax> tatric: yes!
<roaksoax> tatric: how can we help ?
#maas 2016-08-09
<mup> Bug #1589951 changed: why pxe booted node is stucked in commissioning status <MAAS:Expired> <https://launchpad.net/bugs/1589951>
<godleon> Hi all, I can not get the same ip address after power on an power-down suddenly physical machine in MAAS 1.9.3.
<godleon> So the juju agent on it was permanently lost.
<godleon> But it's no problem with the LXC containers on that physical machine.
<godleon> Is there any way to solve this problem? Thanks in advanced.
<godleon> By the way, it seems no such kind of issue in MAAS 2.0 rc3.
<mup> Bug #1611342 opened: UI error while generating a MAAS key (token) <sts> <MAAS:New> <https://launchpad.net/bugs/1611342>
<kuglgruber> hi! i imported a windows image, but i cannot select it for deployment. where can i find any docs or hints??
<mup> Bug #1598301 changed: Deploying centos7 fails during cloud-init networking <MAAS:Invalid> <maas-images:Triaged> <https://launchpad.net/bugs/1598301>
<Sina_Maleki> Hi friends i am on a fresh install of openstack via maas and lanscape but at adding juju there is problem can anyone help me ?
<mup> Bug #1611481 opened: IPMI autoconfiguration and startup not working <MAAS:New> <https://launchpad.net/bugs/1611481>
<mup> Bug #1611481 changed: IPMI autoconfiguration and startup not working <MAAS:New> <https://launchpad.net/bugs/1611481>
<mup> Bug #1611481 opened: IPMI autoconfiguration and startup not working <MAAS:New> <https://launchpad.net/bugs/1611481>
<roaksoax>  /win 4
<mup> Bug #1281758 opened: re-installing maas-region-controller fails <docteam> <MAAS:New> <maas (Ubuntu):Confirmed> <https://launchpad.net/bugs/1281758>
<mup> Bug #1281758 opened: re-installing maas-region-controller fails <docteam> <MAAS:New> <maas (Ubuntu):Confirmed> <https://launchpad.net/bugs/1281758>
<v1k0d3n> hey all...i'm new to maas and been fighting through a condition lately that i can seem to figure out. i'm receiving a "mdadm: CREATE group/user disk/root not found". have others been experiencing this as well?
<v1k0d3n> i can't tell if this is common or if this is something simple i'm overlooking in my deployment.
<v1k0d3n> i verified that the switch isn't doing anything to prevent ports from coming up or anything...
<roaksoax> v1k0d3n: that seems like a curtin issue
<roaksoax> v1k0d3n: what are you doing ? trying to deploy a machine?
<v1k0d3n> yeah, only starting the pxe boot process for a machine.
<roaksoax> v1k0d3n: so the machine PXE boots into the ephemeral environment, and then it fails with that message ?
<roaksoax> during installation?
<v1k0d3n> been trying to research over the last couple of days...but seems like it can go in any direction. figured i'd ask here...just to see if there are any known issues.
<v1k0d3n> pxe appears to load the image and the kernel. then i get that message.
<v1k0d3n> let me start again and give you details.
<v1k0d3n> is there a way to get more debug info?
<roaksoax> v1k0d3n: interesting... so it doesn't even boot into Ubuntu ?
<v1k0d3n> no. doesn't seem so.
<v1k0d3n> should somehting be in the maas logs?
<roaksoax> v1k0d3n: what version is it booting ?
<roaksoax> v1k0d3n: if you can pastebin your logs
<roaksoax> that'd be good
<v1k0d3n> i am really new at maas, so not familiar with the boot process really.
<roaksoax> v1k0d3n: what version of MAAS are you booting ?
<v1k0d3n> 2.0.0 rc2+bzr5156
<v1k0d3n> this is what "comes" with apt 16.04
<roaksoax> strange, i've not seen that before
<roaksoax> but my guess is that it may be a error with the kernel
<roaksoax> v1k0d3n: if you can grab a console log, that'd be great
<v1k0d3n> hmmm
<v1k0d3n> is this logged on the maas server, i guess?
<v1k0d3n> looking in var/log/maas currently
<roaksoax> v1k0d3n: so how are you watching your server boot ?
<v1k0d3n> console. things are kind of going quickly out of view.
<v1k0d3n> it's an infinite loop of mdadm: CREATE group/user disk/root not found
<roaksoax> v1k0d3n: yeah that seems like a kernel bug
<v1k0d3n> so more people are experiencing this as well?
<v1k0d3n> sorry for the questions...is there a recommendation? i've tried this a handful of times with no luck.
#maas 2016-08-10
<v1k0d3n> so the error initially is about bootif: siocgifindex: no such device.
<v1k0d3n> wondering if this could have anything to do with running M.2 drives?
<mup> Bug #1611711 opened: [API] block-device 'add/remove_tag' operations use GET method, not POST <MAAS:Triaged> <https://launchpad.net/bugs/1611711>
<mup> Bug #1611726 opened: Documentation for machine create API doesn't say that power_type is required <MAAS:New> <https://launchpad.net/bugs/1611726>
<mup> Bug #1340920 changed: Extra slash in /etc/apt/sources.list; prevents connecting to some servers <papercut> <trivial> <MAAS:Triaged> <https://launchpad.net/bugs/1340920>
<mup> Bug #1611761 opened: [2.0 RC3 ] Unhandled Error - maasserver.models.regioncontrollerprocess.DoesNotExist <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1611761>
<newmember> I am using MAAS with auto pilot and it only auto detects one node out of 12.  Is there a place to look to help mass find all the nodes?
<newmember> Or get maas to re-try auto discovery?
<roaksoax> newmember: what do you mean by auto-discovery ?
<newmember> when installing maas it appears to look at the network for nodes, it found one.
<newmember> i see all the nodes picked up an IP address form maas
<newmember> from maas
<roaksoax> newmember: maas doesn;t look at the network for nodes
<roaksoax> newmember: if a machine that maas knows nothing about, DHCP's from MAAS
<roaksoax> we will know that /something/ has DHCP'd
<roaksoax> newmember: but typically, MAAS "auto" discovers machines when machines DHCP from MAAS *and* PXE boot from MAAS
<kiko> newmember, you basically need to PXE boot all your 12 nodes against MAAS
<junaidali> Hi everyone, what is the default password that MAAS sets for a node? preseeds master file doesn't specify any password
<junaidali> is there a way that we can use for setting up password for the nodes that we deploy?
<kiko> junaidali, there's no default password -- use the registered ssh key to log in
<kiko> junaidali, the default user on ubuntu is called ubuntu, and on centos is centos
<junaidali> thanks. Is there any setting to set the password for every node that we're provisioning?
<junaidali> password for ubuntu user*
<kiko> junaidali, no, you'll need to do that through a curtin hook. I don't know if I think that's a sane thing to do though
<junaidali> Thanks kiko
<nturner> If I want to add CentOS images to my MAAS setup, is that possible?
<nturner> The docs seem to imply that it is, but don't describe how.
<junaidali> in MAAS 2.0, it is pretty easy. which version of MAAS are you using?
<nturner> junaidali: I'm using 2.0.0~rc3+bzr5180-0ubuntu2~16.04.1
 * nturner sees 2.0.0~rc4+bzr5187-0ubuntu1~xenial1 is now available and updates...
<blake_r> nturner: centos images are in the images page, you can just import them
<nturner> hmmm
<nturner> here's what I see on that page: https://snag.gy/XounFl.jpg
<nturner> Am I tracking the wrong thing? Under Settings my Boot images Sync URL is https://images.maas.io/ephemeral-v2/releases/
<nturner> Which is whatever MAAS defaulted to. Maybe the default has changed for new installs but upgrades don't change it?
<nturner> blake_r: What Sync URL are you using?
<kiko> nturner, can you change releases to daily?
<nturner> sure...
<nturner> well, golly, that was easy!
<nturner> Thanks all.
<newmember> thanks kilo
<newmember> I am using the autopilot deployment from Ubuntu
<newmember> I installed MAAS
<newmember> I created 12 nodes
<newmember> When I look on MAAS all I see is the one node created in MAAS none of the other nodes are created in MAAS
<newmember> I would have thought that they all auto create in MAAS
<newmember> they are all getting DHCP from MAAS
<newmember> Maybe autopilot uses the vmware api to get the 'node' names and load them in MAAS, maybe its not a DHCP discovery thing
<kiko> newmember, you don't need to create the nodes manually
<kiko> newmember, and in fact, I argue you shouldn't
<newmember> kiko, okay
<kiko> newmember, instead, just PXE-boot the machines and see them appear
<kiko> newmember, if they don't appear, you have a problem that needs fixing
<newmember> kiko, I think I have a problem that needs fixing :)
<kiko> newmember, do the machines PXE boot successfully?
<newmember> gets an ip address
<newmember> booting under maas
<newmember> some long line of stuff
<newmember> then stops at "boot:"
<newmember> there is a note:  could not find kernel image
<kiko> newmember, could you get us a screenshot?
<newmember> kiko, great idea
<newmember> http://picpaste.com/Capture-50dZaauj.PNG
<kiko> newmember, yeah, that's definitely odd. what releases do you have selected in your images page?
<newmember> checking
<roaksoax> newmember: it is either you are missing the i386 image , or the import of images hasn't completed
<kiko> newmember, I'm slightly surprised that enlistment is trying to use the i386 image
<kiko> newmember, what hardware are you enlisting?
<newmember> I used the autopilot ova from ubuntu
<kiko> newmember, what's happening is this:
<kiko> - pxelinux's ifcpu64 module is determining your machines are 32-bits only
<kiko> - it then tells MAAS to boot using a 32-bit kernel
<kiko> - the 32-bit images are not yet imported in MAAS, so it fails
<newmember> I think everything has to be 64bit
<kiko> newmember, definitely -- the question is why is this not working in your situation
<kiko> newmember, are these actual machines, or VMs?
<newmember> vms on esxi6
<kiko> ah
<kiko> newmember, and these vms, are they configured to be 32-bit or 64-bit?
<newmember> i imported the ova from ubuntu
<newmember> looking
<kiko> hmmm!
<kiko> newmember, I am a bit clueless about our autopilot vmware install (and vmware) -- so the ova contains the machine definitions already?
<newmember> just on a call, please hang on
<kiko> sure
<newmember> but thanks
<newmember> its 64bit
<newmember> http://picpaste.com/Capture2-ybPaWJdf.PNG
<kiko> newmember, well.. that doesn't actually say the machine is 64-bit
<kiko> newmember, anyway, can you import the i386 images and see if it works?
<kiko> newmember, I need to split but will look through the channel traffic later tonight or tomorrow morning
<newmember> cheers
<roaksoax> newmember: while it may be 64 bit VM, pxelinux may be identifying it as i386
<roaksoax> newmember: i've seen that before
<kiko> dpb1, ping
<kiko> dpb1, have you seen the above?
<roaksoax> not specifically with vmware though
<dpb1> we haven't seen that specifically, no
<dpb1> would be interested to see versions though, if you get to that.
<dpb1> kiko: just so you know, the autopilot doesn't do anything directly with vmware.  we treat it as a regular maas node.
<dpb1> the instructions we have also need to be followed exactly
<kiko> dpb1, right, but do we provide the machine definitions (in machine-readable or doc form), or are they to be manually created?
<mup> Bug #1611949 opened: cryptic error when PXE-boot requires an image not yet imported <MAAS:New> <https://launchpad.net/bugs/1611949>
<mup> Bug #1611949 changed: cryptic error when PXE-boot requires an image not yet imported <MAAS:New> <https://launchpad.net/bugs/1611949>
<mup> Bug #1611949 opened: cryptic error when PXE-boot requires an image not yet imported <MAAS:New> <https://launchpad.net/bugs/1611949>
<mup> Bug #1611980 opened: Upgrade from MAAS 2.0 RC3 to RC4 has pre-removal script error messages <MAAS:New> <https://launchpad.net/bugs/1611980>
<mup> Bug #1611980 changed: Upgrade from MAAS 2.0 RC3 to RC4 has pre-removal script error messages <MAAS:Incomplete> <https://launchpad.net/bugs/1611980>
<mup> Bug #1611980 opened: Upgrade from MAAS 2.0 RC3 to RC4 has pre-removal script error messages <MAAS:Incomplete> <https://launchpad.net/bugs/1611980>
<mup> Bug #1611984 opened: MAAS images page doesn't correctly report image status <angularjs> <notifications> <yui> <MAAS:Confirmed> <https://launchpad.net/bugs/1611984>
#maas 2016-08-11
<mup> Bug #1611999 opened: MAAS 2 does not properly probe power state on Cisco UCS systems <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1611999>
<mup> Bug #1612039 opened: MAAS Cannot Commission <MAAS:New> <https://launchpad.net/bugs/1612039>
<mup> Bug #1612064 opened: [2.0 RC4] Can't add VMware chassis - SSL: CERTIFICATE_VERIFY_FAILED <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1612064>
<mup> Bug #1612064 changed: [2.0 RC4] Can't add VMware chassis - SSL: CERTIFICATE_VERIFY_FAILED <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1612064>
<mup> Bug #1612064 opened: [2.0 RC4] Can't add VMware chassis - SSL: CERTIFICATE_VERIFY_FAILED <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1612064>
<mup> Bug #1612078 opened: [2.0 RC4] __init__ error when entering power parameters for manually enlisted ESXi VM <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1612078>
<newmember> kiki
<newmember> kiko,
<newmember> interesting
<newmember> I am wanting to add all the 'nodes' on an esx server
<newmember> I thought I could add '1' chassis to maas with a filter for all images starting with "node"
<newmember> but
<newmember> Every time I add the esx server as a chassis it adds one node
<newmember> I wonder if its because I copied the same node 12 times?
<newmember> In esx when I created them
<newmember> I am going to import 12 nodes and see what happens
<newmember> deleted all 12 nodes
<newmember> didnt delete the 3 i added to maas, so i deleted them from maas from gui
<newmember> added one node, got an ip address, kernel still missing
<newmember> no node in maas
<newmember> added 6 nodes all with kernel error, none appear in maas automatically
<roaksoax> newmember: check logs, that will tel you what's the issue
<mup> Bug #1595379 changed: 2.0 beta7 : having to recommission some nodes multiple times before they're finally ready  <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1595379>
<mup> Bug #1612064 changed: [2.0 RC4] Can't add VMware chassis - SSL: CERTIFICATE_VERIFY_FAILED <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1612064>
<mup> Bug #1612078 changed: [2.0 RC4] __init__ error when entering power parameters for manually enlisted ESXi VM <oil> <oil-2.0> <MAAS:New> <https://launchpad.net/bugs/1612078>
<mup> Bug #1612203 opened: [Trunk] MAchine unable to pxe with no-such-image while non-related images rae being imported <MAAS:Confirmed> <MAAS 2.0:Incomplete> <MAAS trunk:Confirmed> <https://launchpad.net/bugs/1612203>
<mup> Bug #1612209 opened: can't change image used for commissioning <MAAS:New> <https://launchpad.net/bugs/1612209>
<mup> Bug #1612213 opened: dhcpd leases are not cleaned up properly <MAAS:New> <https://launchpad.net/bugs/1612213>
<mup> Bug #1612209 changed: can't change image used for commissioning <MAAS:Won't Fix> <https://launchpad.net/bugs/1612209>
<mup> Bug #1612209 opened: can't change image used for commissioning <MAAS:New> <https://launchpad.net/bugs/1612209>
<mup> Bug #1612209 changed: can't change image used for commissioning <MAAS:Won't Fix> <https://launchpad.net/bugs/1612209>
<mup> Bug #1612213 changed: dhcpd leases are not cleaned up properly <MAAS:Won't Fix> <https://launchpad.net/bugs/1612213>
<mup> Bug #1612248 opened: MAAS doesn't have debug logging for power actions making it impossible to effectively debug <MAAS:New> <MAAS 2.0:New> <MAAS trunk:New> <https://launchpad.net/bugs/1612248>
<mup> Bug #1612248 changed: MAAS doesn't have debug logging for power actions making it impossible to effectively debug <MAAS:New> <MAAS 2.0:New> <MAAS trunk:New> <https://launchpad.net/bugs/1612248>
<mup> Bug #1612248 opened: MAAS doesn't have debug logging for power actions making it impossible to effectively debug <MAAS:New> <MAAS 2.0:New> <MAAS trunk:New> <https://launchpad.net/bugs/1612248>
<kiko> newmember, did you import the i386 images?
<siva> roaksoax: Hi. I have a question. From MAAS UI I want to select power type as VMware. Now I want to add VMware VM manually to MAAS by providing VMware details(for power) and VMware vm details. Can I do like this? Is it work?
<siva> roaksoax: Because In MAAS UI we can find VMware as power type Option.
<siva> Actually, we have OpenStack autopilot testdrive option for VMware VMs. But I don't want to use Openstack autopilot testdrive. Can I enlist and commissiom VMware VMs as we did for Physical servers.
<siva> roaksoax: Using MAAS
<kiko> siva, good question -- I don't know whether auto-enlistment works on VMs
<kiko> siva, if not, just specify the power details manually
<siva> roaksoax: If we can't do , why we have VMware as power type Option.
<siva> Kiko: Hi. Actually, I manually added VMware vm to MAAS. When I click on "Save changes". Power error install "python-pyvmomi" package on MAAS installed node. I installed . then It started commissioning.
<siva> kiko: Immediately status moved from commissioning to failed commissioning. Failed to power on node - Node could not be powered on: Failed talking to node's BMC for an unknown reason.
<siva> From maas log; It was showing Marking node failed: Node could not be powered on: Failed talking to node's BMC for an unknown reason.
<kiko> looks like the information you've configured MAAS with isn't correct
<siva> Kiko: I have a question here. If we take openstack autopilot testdrive for VMware VMs , How MAAS will manage Power settings and boot settings. I think there we will run autopilot script while running the script it will ask power details....  How we can set boot option as PXE in VMware  vm. I provided power details is correct. Am I enable anything else in vm for power.
<kiko> siva, just a sec
<siva> From VMware vm , I observed  power management : "standby". I am not able to change anything here. But I can enable PXE boot.
<mup> Bug #1612293 opened: [1.9] eth0 tried to get a DHCP IP, no matter what <MAAS:New> <https://launchpad.net/bugs/1612293>
<siva> Hi. if any one tried with the above approach[selecting power type as VMware and moving further instead of openstack autopilot test drive]. Please provide me info.
<kiko> siva, can you send me a screenshot of what you've configured on MAAS for the vmware power driver?
<siva> kiko : Ok. How to send a screenshot in chat?
<siva> Kiko: I don't know how to send a scrrenshot. I send the entered Power details.
<siva> kiko: http://paste.openstack.org/show/554223/
<newmember> kiko, yes I imported the i386 image
<newmember> I am using MAAS Version 1.8.2+bzr4041-0ubuntu1 (trusty1), so I thought I better do a "apt-get upgrade.
<newmember> Maybe I would get maas 2.0
<newmember> I did after ready a few of issues looked to be discussed in 'askubuntu'
<newmember> eg
<newmember> http://askubuntu.com/questions/790768/commissioning-vmware-machines-not-visible-in-maas-using-autopilot-vmware-applian
<newmember> currently its timing out on some of the respositories
<newmember> http://picpaste.com/Capture3-rxtn7viq.PNG
<kiko> oh
<kiko> newmember, you should be using at least 1.9
<siva> kiko: I send the power details and virtual machine details.
<newmember> I downloaded the autopilot fresh from ubuntu just a few days ago
<kiko> newmember, your internet connectivity from the nodes is not working
<kiko> newmember, is the default gateway correct, and is it correctly firewalling/NATting the traffic?
<kiko> newmember, I'm happy you are past the original problem you had though!
<kiko> siva, the VMWare username is really root?
<newmember> kiko, yes yes
<newmember> kiko, well nodes are still not starting
<newmember> and
<kiko> newmember, they aren't starting because they can't get outside
<kiko> newmember, can you dhcp a laptop from the MAAS server and see if it can get to the Internet?
<newmember> kiko, autopilot is 'auto' discovering the nodes
<kiko> newmember, I understand, but if your uplink isn't working, nothing will work
<newmember> the entire subnet can reach the internet and dns works and dhcp yada yada
<kiko> siva, the VMware credentials there are for the vsphere
<kiko> newmember, why is the archive access failing, then?
<newmember> its not responding
<newmember> the first archives responded very quickly
<kiko> newmember, my guess-blame is a problem upstream
<siva> kiko: yes. username is root.
<kiko> newmember, if a laptop is able to do an apt-get update when DHCPing from MAAS, then I would agree I'm wrong
<newmember> http://picpaste.com/Capture4-FkIVKyYu.PNG
<siva> kiko: yes that VMware vSphere client credentials.
<kiko> siva, hmm, interesting. what do your MAAS logs say?
<kiko> newmember, a ping is not an apt-get update :)
<newmember> kiko, hey thanks for even just chatting about it, its best I second guess everything
<newmember> kiko, I can resolve the hostname and reach the host, yes the apt-get has to the next thing.
<kiko> newmember, the apt-get update will tell us if HTTP works
<kiko> siva, could you be encountering this bug? https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1608639
<kiko> siva, check the logs
<kiko> newmember, anyway, you should move from 1.8 to at least 1.9
<siva> Kiko: At starting I got the same issue. Then I manually added vm to maas. Then It was showing   From maas log; It was showing Marking node failed: Node could not be powered on: Failed talking to node's BMC for an unknown reason.
<kiko> siva, check the logs
<siva> Kiko: Actually, I have one Openstack autopilot setup with 5 physical servers. But I should  release two 3 servers for other purpose. Due to this we are planning to use VMware VMs. From my existing MAAS node I just tried to add a new node[that is VMware vm].
<kiko> gotcha
<kiko> siva, I unfortunately don't know enough about vmware to help you, so if you can't check the logs to see what error messages are somebody else will need to look into it
<siva> kiko: I just faced the mentioned issues.
<siva> Kiko: Thanks kiko for your valuable information. I will try with a fresh setup using VMware VMs. Then it will be good.  with respect to your info. I confirmed that we can enlist and commission VMware VMs. I will try this. Thank you.
<kiko> siva, ah, cool -- thanks!
<mup> Bug #1611999 changed: [2.0] MAAS cannot power query with Cisco UCSM power driver <blocks-hwcert-server> <MAAS:Invalid by newell-jensen> <MAAS 2.0:Invalid by newell-jensen> <MAAS trunk:Invalid by newell-jensen> <https://launchpad.net/bugs/1611999>
<newmember> siva, what maas version are you using?
<jwitko> is there any aversion to using MaaS in a virtual machine ?
<jwitko> to be more specific, running a cluster controller in a VM ?
<roaksoax> jwitko: you can use maas in virtual machines
<roaksoax> jwitko: or you can use virtual machines with maas
<jwitko> roaksoax thanks :)
<jwitko> Hey roaksoax, not sure if you're still around but I have a question for HA MaaS setup.   Currently my needs are for a single subnet of < 500 physical machines.  So I was going to run the region controller and cluster controller on the same machine.
<jwitko> if I have two machines running both region and cluster controllers can they be set up in HA?  or does it require the region controller to be separated from the cluster controllers?
<jwitko> and also could the postgres SQL servers also run on those two boxes in HA mode?
<jwitko> not sure if that introduces a risk of data loss in the event of a crash
#maas 2016-08-12
<siva> newmember: I am using  MAAS 1.9.3 for trusty
<newmember> siva, thanks
<newmember> I am just installing MAAS from iso, instead of the autopilot package from the website
<newmember> the version from the ubuntu website was using 1.8
<newmember> okay now I am ver MAAS Version 2.0.0 (rc2+bzr5156)
<newmember> okay
<newmember> when adding vmware chassis I get this error:   Unable to find a rack controller with access to chassis 192.168.0.124
<newmember> I can ping the vmware from maas
<siva> newmember: Yes. At starting I also tried with MAAS from Iso. It was installing the version 1.8.
<newmember> siva, did you add a esx chassis to your environment?
<newmember> with a tcpdump i dont even see the maas host trying to connect to the esx server, no packets are sent
<newmember> hmmmmm
<newmember> dhcpd not running
<siva> newmember: Actually, I didn't try On VMware VMs. I will try it later.
<newmember> two issues now
<newmember> 1.maas-dchp not running
<newmember> 2.cant add vmware esx server to chassis
<newmember> for #2 there is a cert error
<newmember> for #1 still investigating
<siva> newmember:  https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1608639 look at this once for adding certifications
<newmember> root@MAAS:/home/support# maas profile machines add-chassis chassis_type=vmware username=root password=new.pass protocol='https+unverified' hostname=192.168.1.9 prefix_filter=node-
<newmember> failed
<newmember> argument COMMAND: invalid choice: 'profile' (choose from 'login', 'refresh', 'list', 'logout', 'apikey', 'createadmin', 'changepassword')
<newmember> urgh
<mup> Bug #1612620 opened: [API] MAAS server endpoint set_config operation returns OK rather than an object or redirect <MAAS:New> <https://launchpad.net/bugs/1612620>
<Kyoku> can i install a custom freebsd iso on maas?
<kiko> Kyoku, you may be able to, but you'll need to prepare it so curtin can install it
<kiko> Kyoku, curtin basically takes an image and, dd-style, writes it to disk
<kiko> smoser is the local expert
<kiko> newmember, maas login first :)
<kiko> newmember, and the CLI is very confusing, we'll fix it
<mup> Bug #1612662 opened: strange raid(10) layout <MAAS:New> <https://launchpad.net/bugs/1612662>
<mup> Bug #1612668 opened: [2.1 API] devices endpoint 'set_zone' operation returns null on success <MAAS:New> <https://launchpad.net/bugs/1612668>
<smoser> Kyoku, freebsd would be tough.
<smoser> if you could get a ddable image that would boot that'd be the easiest path.
<smoser> theres just not been any time to document or clean up how non-ubuntu curtin installs should work.
<mup> Bug #1612676 opened: [Card sorting] Move DNS to Networks or make it accessible from Networks <MAAS:New> <https://launchpad.net/bugs/1612676>
<mup> Bug #1612680 opened: [Card sorting] Move DHCP configuration on a higher level in Networks or make it accessible from the Networks main page <MAAS:New> <https://launchpad.net/bugs/1612680>
<Kyoku> smoser i have a bottable VMDK that i tried to import into AWS but AWS doesn't support GPT partitions
<Kyoku> smoser so if there's a way to import VMDK into maas that might work if GPT is supported
<smoser> i'm pretty sure there is not such a way.
<Kyoku> seems to be very few cloud services that allow install from iso, and i'm not sure why
<kiko> Kyoku, they all need special hooks to run cloud-init?
<Kyoku> beyond me, i have no idea but to me bare metal as a service should allow me to install 'anything' - isn't that the point of calling it metal as a service?
<smoser> Kyoku, its just very hard to do.
<smoser> there is no standard way to boot an iso over a network
<smoser> and to automate installation of it.
<Kyoku> well it's easy in my head - you upload an iso and give me console to install to disk and it runs in cloud
<Kyoku> lol
<smoser> but even booting that iso is not easy
<Kyoku> ravello do it, but it's stupid expensive and poor performance
<mup> Bug #1612662 changed: strange raid(10) layout <MAAS:Invalid> <https://launchpad.net/bugs/1612662>
<mup> Bug # opened: 1612681, 1612683, 1612685, 1612687, 1612689
<mup> Bug #1612687 changed: Juju not showing real states of deployed nodes via MAAS. <juju-core:New> <https://launchpad.net/bugs/1612687>
<jwitko> Hey All,  if I have a MaaS region/cluster controller setup on a single machine and the server shits the bed what would happen if I was to just create another one in its place with all the same settings?  Servers would reach out to the new DHCP server but it wouldn't know anything about their hostnames or current IPs would it?  Would this be disasterous or is there some sort of mechanism that
<jwitko> comes into play here?
<mup> Bug #1612755 opened: Xenial Deploy fails when using customized interfaces file (curtin, bonding) <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1612755>
<kiko> jwitko, there is no mechanism that comes into play, you'll need to back up the SQL DB
<jwitko> kiko, thanks!
<kiko> jwitko, with MAAS 2.0, you can set up HA components, but you still need to make PostgreSQL HA
<jwitko> yup, i think its time to give 2.0 a shot
<mup> Bug #1229458 opened: grubnetx64.efi tftp client does not work over ipv6 <verification-needed> <MAAS:Confirmed for lamont> <grub2 (Ubuntu):Fix Released by cyphermox> <grub2-signed
<mup> (Ubuntu):Fix Released> <grub2 (Ubuntu Xenial):Fix Committed by cyphermox> <grub2-signed (Ubuntu Xenial):Fix Committed by cyphermox> <https://launchpad.net/bugs/1229458>
#maas 2016-08-13
<sd_> Hi,
<sd_> Is there any way I can insert a script when maas allocate node ?
<geekman_> heeyyy
<geekman_> I have a question, can those be asked here?
<geekman_> ???
<mup> Bug #1612976 opened: Feature Request - Enable MaaS to toggle UID LED via IPMI / iLo <MAAS:New> <https://launchpad.net/bugs/1612976>
#maas 2017-08-07
<BlackDex> good morning
<BlackDex> i'm having some trouble with commisioning of a few nodes
<BlackDex> ureadahead:/etc/machine-id: Error retrieving chunk extents: Operation not supporte
<BlackDex> getting messages like that all over the log
<mup> Bug #1708925 changed: [2.2] ambiguous error when pod missing pool definition: int() argument must be a string, a bytes-like object or a number, not 'NoneType' <MAAS:New> <https://launchpad.net/bugs/1708925>
<mup> Bug #1709095 opened: Commissioning fails when cloud init can't find a data source <MAAS:New> <https://launchpad.net/bugs/1709095>
<mup> Bug #1709095 changed: Commissioning fails when cloud init can't find a data source <MAAS:New> <https://launchpad.net/bugs/1709095>
<mup> Bug #1709095 opened: Commissioning fails when cloud init can't find a data source <MAAS:New> <https://launchpad.net/bugs/1709095>
#maas 2017-08-08
<bashlakov> Hi! Can anybody consult me about commercial support? I sent form from website, but employee that answer the first message didn't answer anymore
<rbasak> roaksoax: ^
<mup> Bug #1709284 opened: curtin does not put mount_options passed by MAAS into /etc/fstab <curtin:New> <MAAS:New> <https://launchpad.net/bugs/1709284>
<mup> Bug #1709284 changed: curtin does not put mount_options passed by MAAS into /etc/fstab <curtin:New> <MAAS:New> <https://launchpad.net/bugs/1709284>
<mup> Bug #1709284 opened: curtin does not put mount_options passed by MAAS into /etc/fstab <curtin:New> <MAAS:New> <https://launchpad.net/bugs/1709284>
<dancn> hello, I new to mass (first install), and I do not see the next action... I followed the doc and now I have one Region and Rack controller with the status â, all the services in the rack controller are in â and there is no hints to restart/reconfigure them! Thanks in advance for your suggestion
<exoduswtf> Does anyone know of a way to disable the archive.ubuntu.com repos and use custom repos during deployment?
<exoduswtf> I tried adding apt sources to the Curtin preseed, but nodes are still trying to reach out to archive.ubuntu.com
<exoduswtf> And you can't disable that repo in the UI
<pmatulis> exoduswtf, https://docs.ubuntu.com/maas/2.2/en/manage-repos ?
<pmatulis> or, ok, those cannot be disabled?
<exoduswtf> Correct
<exoduswtf> Would like to use an internal mirror without needing outbound access
<pmatulis> hmm
<pmatulis> dancn, not sure i follow. what instructions are you following and what are you expecting?
<pmatulis> exoduswtf, maas is supposed to be offline-ready
<dancn> pmatulis: instructions at https://maas.io/install; now I have purged all mass and reinstalled, and now I have a different problem: the rack controller is not auto created and maas-rack register (on the same host) do not have effect (the contoller show only "Region controller") even after a reboot
<dancn> making progress: spotted in the log: Region not available: An error occurred while connecting: 113: No route to host... seem strange since they are on the same host... I will dig deeper...
<pmatulis> dancn, yeah. those aren't really the docs. not that it will necessarily solve your problem but try looking at the doc URL above
<exoduswtf> pmautils, that hasn't been my experience...but I might be missing something obvious
<dancn> pmatulis: spotted the problem: the cut and paste from the page MAAS/#/nodes?tab=controllers after clicking "Add rack controller" contains the wrong region controller IP!!!
<exoduswtf> I'm able to sync the images from an internal mirror, but the builds seem to still reach out to external repos
<pmatulis> dancn, good stuff
<pmatulis> exoduswtf, right. i'm going to look into it
<pmatulis> exoduswtf, can you confirm that there is no way to *edit* the 'Ubuntu archive' URL under the 'Package repositories' tab?
<pmatulis> exoduswtf, also, what version of MAAS are you running?
<exoduswtf> Version 2.2.1
<exoduswtf> Editing the default repo seems reasonable...
<pmatulis> exoduswtf, so it works?
<exoduswtf> Let me test it and see
<pmatulis> thanks
<exoduswtf> Thank you
<pmatulis> works?
<exoduswtf> Works
<exoduswtf> Told ya it was probably something obvious
<dancn> hmm, I still need a little help... the rack controller seems ok (everything green) but I am not able to configure the subnets and dhcp... something somewhere must be wrong...
<dancn> for example in the interface table on the rack controller i see the line:
<dancn> enp2s0f0 Physical fabric-3 untagged 10.123.0.0/24 (t1) 10.123.0.1 (Static assign)
<exoduswtf> Pmautils - Thanks for the help
<dancn> but the "ip addr show enp2s0f0" does not show the assigned ip: 10.123.0.1
<dancn>         (Static assign)  [16:40]
<dancn> <exoduswtf> Pmautils - Thanks for the help
<dancn>  
<dancn> I also see the incoming dhcp request with tcpdump but nobody replies
<dancn> ok, after setting the ip manually on the interface with "sudo ip addr del 10.20.0.1/32 dev enp1s0f1" the dhcp started responding...
<pmatulis> exoduswtf, in terms of UX then, i take it that it was not clear that the URL could be changed?
<pmatulis> exodusftw, welcome
<sck> hi friends!
<sck> I am try using cloud-init at image CentOS, but I can not work like used curtin_userdata parameters the cloud-init.  Does anyone know about this it working with CentOS?
<mup> Bug #1709284 changed: curtin does not put mount_options passed by MAAS into /etc/fstab <curtin:New> <MAAS:Invalid> <https://launchpad.net/bugs/1709284>
<mup> Bug #1709482 opened: removing filesystem resets partition name and filesystem <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1709482>
<mup> Bug #1709482 changed: removing filesystem resets partition name and filesystem <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1709482>
<mup> Bug #1709482 opened: removing filesystem resets partition name and filesystem <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1709482>
#maas 2017-08-09
<bladernr> Hey, is there a way to tell maas to only allow user Foo to have access to specific nodes?
<bladernr> so that Foo can only see and manipulate those nodes when they log in?
<pmatulis> i don't think so, other than having the admin acquire all nodes by default and then acquire some nodes to a specific user
<pmatulis> (acquire == allocate)
<mup> Bug #1706462 opened: juju tries to acquire machines in specific zones even when no zone constraint is specified <cdo-qa> <foundations-engine> <juju:Triaged> <MAAS:New> <https://launchpad.net/bugs/1706462>
#maas 2017-08-10
<sncder> hiï¼guys
<mup> Bug #1709850 opened: Can't enlist Huawei node with MaaS 2.2.1 <MAAS:New> <https://launchpad.net/bugs/1709850>
<mup> Bug #1709850 changed: Can't enlist Huawei node with MaaS 2.2.1 <MAAS:New> <https://launchpad.net/bugs/1709850>
<mup> Bug #1709850 opened: Can't enlist Huawei node with MaaS 2.2.1 <MAAS:New> <https://launchpad.net/bugs/1709850>
<bryan_att> hi anyone - are there any special tricks to get MAAS 2.2.2 to deploy Centos7 ? every time I try this, the OS is installed but (1) there are no network interfaces configured (though I have defined them and set them to auto assign); (2) there are no users created that I can tell of; I see a video of one guy doing this but the details are sketchy https://www.youtube.com/watch?v=yCZNSyLHpcQ
<mup> Bug #1709907 opened: Interface alias name causes invalid DNS FQDN <sts> <MAAS:New> <https://launchpad.net/bugs/1709907>
<bryan_att> if anyone on this channel is regularly deploying Centos7 with MAAS, could you share the details of how you have MAAS setup so I can understand any dependencies?  I can't find any docs on what to do if the issues I am finding occur, or any MAAS setup requirements for Centos7 deployment.
<jcordeiro> hi there.
<jcordeiro> im using MAAS Version: 2.2.2-6099-g8751f91-0ubuntu1~16.04.1
<jcordeiro> my machines are virtual(libvirt), and MAAS is able to power them on and off
<jcordeiro> i have installed my machines with maas but now i want to reinstall them
<jcordeiro> my machines are failing to comission
<jcordeiro> what boot setup should i use? pxe into all?
<jcordeiro> any pointers for me to try?
<pmatulis> jcordeiro, PXE should always be enabled on the libvirt guests
<pmatulis> jcordeiro, to reinstall: 'Release' and 'Deploy'
<bryan_att> does anyone have links to the details of how to format the curtin preseed files, and what are the available preseed operations? All I can find on MAAS documentation is a few examples.
<mup> Bug #1709284 opened: curtin does not put mount_options passed by MAAS into /etc/fstab <ci> <curtin:Confirmed> <MAAS:Triaged by blake-rouse> <https://launchpad.net/bugs/1709284>
#maas 2017-08-11
<mup> Bug #1589140 opened: No WOL option in latest MAAS version for 16.04 also the Manual settings crashes <MAAS:Confirmed> <https://launchpad.net/bugs/1589140>
<reikermann> Hi, my MAAS hardware test stress-ng-cpu-long is getting a timed out error after 11:59:26 , but the test definition is a 12h test why is the timeout so short?
<mup> Bug #1710092 opened: Hardware Tests have a short timeout  <MAAS:New> <https://launchpad.net/bugs/1710092>
<mup> Bug #1710096 opened: [2.2] for non-local network ranges, allow_proxy should be false by default <MAAS:New> <https://launchpad.net/bugs/1710096>
<D4RKS1D3> bryan_att, you have the curtin documentation
<D4RKS1D3> let me share with you the link
<D4RKS1D3> bryan_att, https://media.readthedocs.org/pdf/curtin/latest/curtin.pdf
<mup> Bug #1710177 opened: curtin in-target commands do not work for ubuntu core <MAAS:New> <https://launchpad.net/bugs/1710177>
<bryan_att> D4RKS1D3: many thanks, just what I was looking for. I'm trying to find out how to circumvent the cloud-init/NIC race condition that resulted in the DEVTIMEOUT option being added, so I can avoid the 5-min hang due to the primary NIC not being ready in time.
<bryan_att> e.g. per https://bugzilla.redhat.com/show_bug.cgi?id=1194623 and https://lists.stg.fedoraproject.org/archives/list/spins@lists.stg.fedoraproject.org/thread/2OYS7FSN6WUFBF7CQG4STCR73K72QUPK/  - I think that is what I am encountering.
<bryan_att> anyone familiar with how to format the kernel options that MAAS allows: can you tell me if the format at http://cloudinit.readthedocs.io/en/latest/topics/network-config.html for disabling network configuration (network-config={config: disabled}) is correct for use in MAAS?
<bryan_att> I am trying to see if disabling the cloud-init network config can help me avoid the NIC ready race condition that causes a 5-min delay in server bootup
<mup> Bug #1710241 opened: Cannot deploy customized ubuntu cloud image <MAAS:New> <https://launchpad.net/bugs/1710241>
<mup> Bug #1710241 changed: Cannot deploy customized ubuntu cloud image <MAAS:New> <https://launchpad.net/bugs/1710241>
<mup> Bug #1710241 opened: Cannot deploy customized ubuntu cloud image <MAAS:New> <https://launchpad.net/bugs/1710241>
<maticue> hi everyone! one question, if I add a new Bare Metal to my network and I boot it at first time with PXE, MAAS will discover it. My question is, at that stage is it possible to execute directly the commissioning stage automatically?
<maticue> I'm trying to identify automatically IPMI
<maticue> for each bare metal
<mup> Bug #1710278 opened: [2.3a1] named stuck on reload, DNS broken <MAAS:New> <https://launchpad.net/bugs/1710278>
<blizzow> I have three machines that maas has enlisted.  I changed the hostnames/entered their IPMI information and selected all of them to be commissioned.  They all show the RAM and CPU core after being commissioned, but there are no disks detected. What do I have to do to get the disks detected and do a deployment?
#maas 2017-08-12
<mup> Bug #1710308 opened: [2.2] MAAS should throttle bind reloads <MAAS:Triaged> <MAAS 2.2:Triaged> <https://launchpad.net/bugs/1710308>
<mup> Bug #1710310 opened: [2.2] DNS service monitoring should issue test queries <MAAS:Triaged> <https://launchpad.net/bugs/1710310>
<mup> Bug #1588466 changed: gpg --batch --verify during maas install causes Unhandled Error <cdo-qa> <MAAS:Expired> <simplestreams:Incomplete> <https://launchpad.net/bugs/1588466>
<mup> Bug #1588466 opened: gpg --batch --verify during maas install causes Unhandled Error <cdo-qa> <MAAS:Expired> <simplestreams:Incomplete> <https://launchpad.net/bugs/1588466>
<mup> Bug #1588466 changed: gpg --batch --verify during maas install causes Unhandled Error <cdo-qa> <MAAS:Expired> <simplestreams:Incomplete> <https://launchpad.net/bugs/1588466>
<maasCluster> I've just recently setup MAAS Region controller and cannot get additional MAAS nodes to remain on during boot up... Get a power error each time, and the nodes then turnoff after getting commands from MAAS controller
<maasCluster> Can anyone help please
<maasCluster> I occasion can get one or two of the nodes to stay on (green on, and ready state)... but then when trying to get the 3rd node on, everything powers off
<jcordeiro> is there a known bug that makes my MAAS machine go 100% cpu on idle?
<jcordeiro> jesus lutostag ... thats some bad connection you have
<jcordeiro> with 8 cpus my load is 14
<jcordeiro> the thing using all my cpu is:
<jcordeiro>  9332 maas      20   0 1522636 190196  19832 R  42.4  1.2 666:15.00 twistd3
<jcordeiro> https://bugs.launchpad.net/maas/+bug/1558314 ...
<jcordeiro> https://bugs.launchpad.net/maas/+bug/1705518
<jcordeiro> lol, that was fast on lutostag
#maas 2017-08-13
<mup> Bug #1696276 changed: Comissioning scripts should trigger a hardware reinventory <MAAS:Expired> <https://launchpad.net/bugs/1696276>
<marlinc> Is it possible to use MAAS purely to write a Ubuntu cloud image to the server using PXE but use another cloud-init datasource that I'm planning on using for virtual machines as well?
#maas 2018-08-06
<bdx> hello
<bdx> I need to remove a stuck controller
<bdx> I think I need to do it from the db directly as I cant seem to do it from the gui
<bdx> or cli
<bdx> I'm wondering what table I should be deleting the controller entry from?
<roaksoax> bdx: is this the master controller ?
<bdx> no
<bdx> its a non existing controller
<bdx> that wont leave
<bdx> leaving my maas borked for some time now
<bdx> I need to move forward today
<bdx> I feel I need to just remove the entry from the nodes table
<roaksoax> bdx: why won't it just remove ?
<bdx> its this https://bugs.launchpad.net/maas/2.4/+bug/1774538
<bdx> I've tried this https://paste.ubuntu.com/p/kvT75jKWVm/
<bdx> but it just doesnt seem to make sense
<bdx> postgresql seems to be telling there is an entry that exists that doesnt exist
<roaksoax> bdx: update the 'pool' to 'None' for that uuid and try deleting it
<roaksoax> ?
<bdx> roaksoax: the 'pool_id' field in the 'maasserver_node' table?
<roaksoax> bdx: uhmm i guess so since you are looking at the db
<bdx> roaksoax: where did you mean?
<bdx> roaksoax: https://paste.ubuntu.com/p/B6hhKHn5RX/
<roaksoax> bdx: so what I saw the fix wasw doing was:
<roaksoax> +    Node = apps.get_model('maasserver', 'Node')
<roaksoax> +    Node.objects.exclude(node_type=NODE_TYPE.MACHINE).update(pool=None)
<bdx> yeah I saw that too
<bdx> deleting the controller from the gui
<bdx> I get a different result then I previously did
<bdx> now that the pool_id = NULL
<bdx> it doesnt error anymore
<bdx> but now it just sits https://imgur.com/a/7HZacVx
<bdx> lol
<bdx> worse then an eerror
<bdx> oh shoot
<bdx> I spoke too soon
<roaksoax> bdx: may be trying to power off
<bdx> its removed
<roaksoax> bdx: cool
<bdx> yes
<bdx> thats great
<bdx> roaksoax: thanks for taking time
<bdx> appreciated
<roaksoax> no prob, glad to help
<roneth99> How to force MAAS to deploy an image no matter an existing image on the disk?
<roneth99> It appeared that MAAS skipped deploying because it sees there is already an existing OS on the disk.
<roaksoax> roneth99: ??? MAAS doesn't skip deploying if there's a pre-installation. MAAS even tells the partitioner to wipe the superblock of all disks
<bdx> geh, seems I have a few dangling issues
<bdx> https://imgur.com/a/nNL36fM
<bdx> seems now the proxy and chrony wont start
<bdx> roaksoax: starting to feel like I should switch back to the deb now, possibly these things are snap related?
<roaksoax> bdx: it would be matter of finding out why they dont start, but that said, should work unless something is conflicting e.g. other services inside the actual machine are owning those ports /
<roaksoax> ?
<roneth99> roaksoax: Monitoring the console, I can see the "Ubuntu" image got laid out... Run its things.. and eventually kicking off Cloud-Init, then finally reboot the system. But the system came back with old OS....
<bdx> https://paste.ubuntu.com/p/2n8TGczTv7/
<bdx> no maas apt packages are installed https://paste.ubuntu.com/p/8TPhMs3hJT/
<bdx> il pick at this for a while and see what I can come up with
<roneth99> roaksoax: By the way, what did you mean by "pre-installation"... Where can I check that if that exists.
<roaksoax> bdx: check the logs on those services
<roaksoax> roneth99: the only thing i could think of why something like that would happen is that another disk has the installation and the system is booting into that other disk
<roaksoax> roneth99: but that would be weird, it would be helpful to have logs to better understand what's going on
<bdx> my  chrony.log is spammed with
<bdx> 2018-08-06T18:40:00Z chronyd version 3.2 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SECHASH +SIGND +ASYNCDNS +IPV6 -DEBUG)
<bdx> 2018-08-06T18:40:00Z Fatal error : Another chronyd may already be running (pid=1291), check /run/chronyd.pid
<bdx> ok
<bdx> so that makes sense
<bdx> what you said
<bdx> ahhh looks like I had dangling squid and chrony from the deb install
<bdx> that makes sense
<bdx> ok
<bdx> thanks again!
<roneth99> roaksoax: I am erasing the disk and will attempt another try.... If I see the same issue again, what logs would be usefule for you? maas.log, rackd.log, regiond.log ?
<xygn4l> roaksoax: plans to retire IPMI and provide redfish for Dell? not comfortable using protocol with an open cve since 2003.  security teams REALLY hate it.
<roaksoax> xygn4l: as of today, we have no plans, unless of course it comes through customers
#maas 2018-08-07
<naturalblue> hey. is there anyone here who might answer me a question on maas with juju lxds and maas's builtin proxy feature
<naturalblue> I have setup maas and juju. I have multiple VLANS and all of them have a gateway address on the maas controller. I have enabled builtin proxy but when I juju deploy an lxd container the proxy settings are not configured and i have to manually enter the lxd and create/add the maas proxy settings into apt.d/proxy.conf file
<naturalblue> Is maas not supposed to pass down proxy settings to machines/lxds
<xygn4l> roaksoax: commission bug updated
<xygn4l> roaksoax: also, is it normal for rack controllers to do their probe for external DHCP servers check, and find its partner rack controllers IP each time?   should both nodes be answering DHCP?
<xygn4l> roaksoax: i see 5 services running on both nodes,  rackd, http, tftp, dhcpd, and ntp all enabled on both
<mpontillo> naturalblue: you might reach out in #juju; it's likely that when the container is deployed, the proxy settings from MAAS aren't being "inherited" (since juju operates with more clouds than just MAAS, it may not have been clear that this was a requirement)
<roaksoax> xygn4l: if you could, please file a bug for that
<xygn4l> re: ipmi?
<roaksoax> xygn4l: the dhcp issue above
<xygn4l> interesting.  not related to our commission ticket?
<roaksoax> xygn4l: it may actually be, if you can provide that update on the bug report, it would be good, and we can later split it
<roaksoax> xygn4l: for ipmi replied on pm
#maas 2018-08-08
<naturalblue> mpontillo: Thanks, I wll ask them.
<boritek> hello, I am trying to install MAAS from Ubuntu Server 18.04.1 iso in Virtualbox, but it fails
<boritek> through PXE
<xygn4l> roaksoax:  bug 1786127 filed for DHCP issues.  commission bug referenced inside.  what info do you want in it?
#maas 2018-08-09
<icey> roaksoax you mentioned that you thought that https://pastebin.ubuntu.com/p/Gf3qkQ6SzS/ was a cloud-init bug, resolved with the cloud-init in -proposed; do you have any bug reference that I can track, or further details about when you think the fix will be released? I run into it with the Juju snap in both candidate and edge, but not the stable vers
<icey> ion (I run into a different juju bug semi-regularly there)
<roaksoax> icey: i would need to look through my email thread and see if I find it
<icey> no hurry roaksoax
<icey> although I'd appreciate it :-D
<ananke> how does one clear failed tests, other than re-running them?
<roaksoax> ananke: there's an option to override failed tests
<ananke> roaksoax: unfortunately, I am unable to find it. eg: https://imgur.com/a/yVC8HQ7
<ananke> there is nothing in the context menu for the machine, specific test options just show 'hide metrics' or 'view previous t...'
<ananke> I can mark the entire node as broken, then as fixed, but that seems to have no impact on individual tests
<roaksoax> ananke: the machine is ready so you have no option
<roaksoax> overrideing failed tests wont clear those tests that failed
<roaksoax> would make the machine ready but still show there was a failed tests
<ananke> right, so how does one clear those warnings?
<ananke> I have tests that show as timed out, like that badblocks-destructive, and ones that show skipped (such as the smartctl-validate, because those are raid arrays). since the results of those tests are irrelevant, it would make sense to remove them
<roaksoax> ananke: there's no way to force remove those, unless you actually remove them from the DB
<ananke> roaksoax: that's a wee bit disappointing, perhaps I'll file it as a feature request on github
<roaksoax> ananke: we dont track bugs on gh
<roaksoax> ananke: we track them on lp
<roaksoax> :)
<roaksoax> ananke: but you can start a ux discussion on discourse.maas.io
<ananke> ahh, thank you
<ananke> today I discovered another thing that I may have to suggest: ability to disable ipv6 for squid
<ananke> if your dns server returns AAAA records, yet you don't have ipv6 address, doing anything with built-in proxy is a crapshot
#maas 2019-08-05
<mup> Bug #1838942 opened: Websocket API needs support for users.[create,update,delete] and userProfile.update <MAAS:New> <https://launchpad.net/bugs/1838942>
<mup> Bug #1838942 changed: Websocket API needs support for users.[create,update,delete] and userProfile.update <MAAS:New> <https://launchpad.net/bugs/1838942>
<mup> Bug #1838942 opened: Websocket API needs support for users.[create,update,delete] and userProfile.update <MAAS:New> <https://launchpad.net/bugs/1838942>
<mup> Bug #1838943 opened: Cannot PXE boot arch 0f due to protocol mismatch <MAAS:New> <https://launchpad.net/bugs/1838943>
<mup> Bug #1838942 changed: Websocket API needs support for users.[create,update,delete] and userProfile.update <MAAS:New> <https://launchpad.net/bugs/1838942>
<mup> Bug #1838943 changed: Cannot PXE boot arch 0f due to protocol mismatch <MAAS:New> <https://launchpad.net/bugs/1838943>
<mup> Bug #1838942 opened: Websocket API needs support for users.[create,update,delete] and userProfile.update <MAAS:New> <https://launchpad.net/bugs/1838942>
<mup> Bug #1838943 opened: Cannot PXE boot arch 0f due to protocol mismatch <MAAS:New> <https://launchpad.net/bugs/1838943>
<mup> Bug #1838517 changed: Unable to deploy P8 node "modoc" after migrated to power8-maas <ppc64el> <MAAS:Fix Released> <The Ubuntu-power-systems project:Fix Released by maas> <https://launchpad.net/bugs/1838517>
#maas 2019-08-06
<ivve> hey, i have a wierd problem. i followed the HA guide for regiond but encountering this issue:
<ivve> 2019-08-06 11:26:23.198 UTC [16335] maas@maasdb DETAIL:  Password does not match for user "maas".
<ivve>         Connection matched pg_hba.conf line 94: "host    all             all             ::1/128                 md5"
<ivve> anyone knows that this could be? i tried editing pg_hba.conf to add all on both servers but doesn't seems to change anything
<mup> Bug #1839170 opened: Add support for IPMI system event logs <MAAS:New> <https://launchpad.net/bugs/1839170>
<mup> Bug #1839188 opened: [2.6] [UI] Power type of auto-enlisted machine could not be changed to "Manual" <MAAS:New> <https://launchpad.net/bugs/1839188>
<mup> Bug #1839189 opened: [2.6] Malformed input in the IP addr field in "power parameters" causes part of SQL error to be shown <MAAS:New> <https://launchpad.net/bugs/1839189>
<mup> Bug #1839188 changed: [2.6] [UI] Power type of auto-enlisted machine could not be changed to "Manual" <MAAS:New> <https://launchpad.net/bugs/1839188>
<mup> Bug #1839189 changed: [2.6] Malformed input in the IP addr field in "power parameters" causes part of SQL error to be shown <MAAS:New> <https://launchpad.net/bugs/1839189>
<mup> Bug #1839188 opened: [2.6] [UI] Power type of auto-enlisted machine could not be changed to "Manual" <MAAS:New> <https://launchpad.net/bugs/1839188>
<mup> Bug #1839189 opened: [2.6] Malformed input in the IP addr field in "power parameters" causes part of SQL error to be shown <MAAS:New> <https://launchpad.net/bugs/1839189>
<mup> Bug #1839227 opened: manually added dns records are deleted <MAAS:New> <https://launchpad.net/bugs/1839227>
<mup> Bug #1839227 changed: manually added dns records are deleted <MAAS:New> <https://launchpad.net/bugs/1839227>
<mup> Bug #1839227 opened: manually added dns records are deleted <MAAS:New> <https://launchpad.net/bugs/1839227>
#maas 2019-08-07
<Japje> can someone help me out a little bit. I've added a custom image (debian-10) and i want to deploy a machine via the cli using the custom image. However it kindly refuses me, and i think im just overlooking something simple
<Japje> image was added with: ~# maas admin boot-resources create name=custom/debian title="debian-10.0.2" architecture=amd64/generic content@=/root/debian-10.0.2.tgz
<Japje> and now im trying to deploy it with: ~# maas admin machine deploy gh48yd distro_series="custom/debian-10.0.2"
<Japje> but im getting 'custom/debian-10.0.2' is not a valid distro_series.
<Japje> ok, found it, turns out the "custom/" part of the name is not needed when doing the deploy, and just doing distro_series="debian" is sufficient
<roaksoax> Japje: filetype
<roaksoax> Japje: look at the help for filetype
<ivve> anyone else upgraded to maas 2.6.0 and having issues with PXE-E53 no boot filename received? i have two environments, one works other doesn't. tried restarting, removing dhcpd.conf and lease files which seems to correctly indicate to serve a FNAME in the BOOTP request. however dhcpdump shows 90% of the BOOTPREPLY is empty,  FNAME: .
<ivve> worked before upgrade, stopped working post upgrade
<ivve> also the same bootpreplies seem not to reflect updates i do in maas, i.e switching around DNS server, NTP or other
<roaksoax> blake_r: ^^
<ivve> tried restarting everything, even reboots (well had to try). tried looking into some tables in the DB but can't find anything relevant..
<ivve> i did the upgrade in the test environment and also tried HA, works without any remarks
<ivve> typical :P
<ivve> purged leases file and dhcpd.conf while restarting dhcp just in case. but they are recreated properly with the issue still present
<blake_r> ivve: so the dhcpd.conf is created and the maas-dhcpd service is running?
<ivve> blake_r: yup
<blake_r> ivve: can you send me a copy of the dhcpd.conf contents?
<ivve> sure
<ivve> blake_r: any particular part you are looking for?
<blake_r> ivve: just that all the appropraite configuration values are correct for the file
<blake_r> ivve: you can pm it to me, if you prefer
<ivve> aye working on it :)
<ivve> blake_r: sending now, i redacted some domains and all hosts, i guess they are not required
<ivve> blake_r: https://pastebin.com/UARxPSDA
<ivve> hastebin seems down or smth
<ivve> so this one had to do
<ivve> blake_r: i guess this one is a bit better http://paste.openstack.org/show/755629/
<ivve> ill be back a bit later, need to drive home
<humbolt> Hi there! Does MAAS support bootable RAID1 configuration? I see that I can configure a RAID device, but will the MBR and optional efi boot partition be written to both disks?
<roaksoax> humbolt: it should
<roaksoax> rharper: ^^
<roaksoax> blake_r: ^^
<roaksoax> 4/win 6
<blake_r> humbolt: legacy booting yes, efi will only point to one of the disks, but you can use an efi shell to fallback to the other manually
<rharper> roaksoax: we do support raid1 boot, for efi boot partition, there's no acceptable way that I know of to ensure that /boot/efi is mirrored and in-sync; and grub2 itself I don't think has any raid1 support for setting up a mirror before reading/using; lastly, not sure that efi firmware won't write data to the efi boot partition that won't get clobbered if you did mirror efi boot
<rharper> for all cases, we install grub to each device in the raid mirror;  for legacy/mbr this all just works;  for efi, it's an exposure due to the above challenge
<ltrager>  /boot/efi has to be readable by the UEFI firmware. I think adding Linux software RAID would break that
#maas 2019-08-08
<Japje> roaksoax: thanks, will do
<Japje> regarding debian as an custom image, im having trouble with deployment. The curtin_userdata_custom flag for preserve_sources_list seems to be getting ignored
<Japje> and its overwriting the debian repo's with ubuntu repos
<Japje> which ofcourse results in a failed deployment
<tosaraja> I have MAAS 2.6.0 and a Ubuntu 18.04.2 server as a virtual machine getting an IP address from MAAS. But the DNS records in MAAS don't get the data from the Ubuntu VM. Sometimes restarting MAAS helps, but not always. How does the workflow go as in updating the DNS records? Is it the VM that should report it right after receiving the IP from the DHCP server?
<mup> Bug #1839430 opened: Add an enum for node power types <MAAS:New> <https://launchpad.net/bugs/1839430>
<mup> Bug #1839430 changed: Add an enum for node power types <MAAS:New> <https://launchpad.net/bugs/1839430>
<mup> Bug #1839430 opened: Add an enum for node power types <MAAS:New> <https://launchpad.net/bugs/1839430>
<ivve> blake_r: hey man, did you get any chance to look at those configuration files i sent you?
<tosaraja> seems my problem has happend before as well: https://bugs.launchpad.net/maas/+bug/1761326
<tosaraja> 2.6 isn't having any of those "maasserver.rpc.leases" outputs in regiond.log
<tosaraja> journal does however see that maas' dchp server gives out the ip addresses and renews leases. it's just that the dns server knows squat about what's going on
<ivve> anyone here have any clue if its possible to downgrade from 2.6.0 to 2.4.2?
<tosaraja> ivve: you make that sound like i should stop my work on trying to upgrade from 2.4.2 to 2.6.0 :P
<ivve> tosaraja: well lots of problems with pxe
<ivve> i have test and prod env. it worked in test but not in prod
<ivve> so really confusing
<tosaraja> oh i haven't gotten that far :D i can't get the DNS server to understand that the dhcp server just gave my opennebula server an andress so that it would update the dns record. it remains empty
<tosaraja> pxe is only coming up after i get that part working
<ivve> dns and dhcp seems to work no issues
<ivve> and well to be honest pxe works but gives incorrect values
<ivve> i.e. doesn't pass FNAME
<tosaraja> well it's not the first time i solve issues by simply restarting maas. I think i'll try that _again_
<tosaraja> i didn't get any of "maasserver.rpc.leases" in regiond.log before restarting maas. now i do...perhaps it began working
<tosaraja> it works! so 3 restart in 2 days fixed my problem. now on to the pxe myself :D
<ivve> anyone knows what this could be? https://pastebin.com/raw/rxQJhZG3
<ivve> tcpdump shows 3x udp packets go back and forth to the LOM
<ivve> i can reach it, so no issues with communications on any ports
<blake_r> ivve: sorry I did not
<blake_r> ivve: let me take a look now
<ivve> blake_r: you have the config in priv or here
<ivve> blake_r: i also noticed that it seems like there is an error in the BMC count (i think). since i have good communication tcp/ip wise but BMC doesn't talk with rackd as per https://pastebin.com/raw/rxQJhZG3
<blake_r> ivve: looking at the DHCP config I only see the bootloaders defined in 1 of the vlans and not all of them
<blake_r> ivve: do you know which subnet your machine is PXE booting from?
<ivve> dhcp and pxe is in 1 vlan (108) and helper to the other vlans
<ivve> so that should be correct?
<blake_r> ivve: whats the subnet?
<ivve> 10.22.5.0/24
<blake_r> yeah that one seems okay
<blake_r> that has the bootloaders defined
<ivve> and i have helpers to 10.22.5.20 & 10.22.5.22 (we setup a new maas)
<ivve> in HA
<blake_r> yeah I see the HA
<ivve> since we had issues with bnx2 drivers
<blake_r> what does the BIOS give when PXE booting?
<ivve> these are now virtual
<ivve> just that FNAME is empty
<ivve> PXE-E53 error
<ivve> which it is, when dhcpdumping on the maas node(s)
<ivve> i can see: FNAME: .
<ivve> instead of expected: FNAME: lpxelinux.0.
<ivve> we tested booting up both physical and virtual machines with just live images to test dhclient and got IP's from maas with no issue
<ivve> we have multiple vendors with different nic, i.e. hp or dell and nics like broadcom, intel. which by the way worked 2.4.x but not now (exactly the same machines, just redeployed)
<ivve> blake_r: i guess i can offer you a sosreport, if you think you need one?
<blake_r> ivve: can you turn off DHCP on all the other VLAN's
<blake_r> ivve: and just have it on for that 1 valn
<blake_r> ivve: i believe isc-dhcp is giving you an IP address from another subnet that doesn't have the bootloaders defined
<blake_r> ivve: you have 5 other vlans with DHCP enabled, turn those off and try that machine again
<ivve> blake_r: okay going for that
<ivve> blake_r: turned off all dhcp to start with, enabling on the native vlan first from the maas
<ivve> systemctl status maas-dhcpd.service confirms everything is off
<ivve> blake_r: same error.. :(
<ivve> meaning regiond spits out https://pastebin.com/raw/rxQJhZG3
<blake_r> ivve: dhcpd should be running for that one vlan
<blake_r> ivve: can you provide new paste of dhcpd.conf
<ivve> yes
<ivve> coming up
<blake_r> ivve: maas doesn't use DHCP enable status to check if a rack controller can query a BMC
<blake_r> ivve: did you disable DHCP on the vlans? or delete the vlans?
<ivve> disable dhcp
<ivve> did not delete the vlans
<ivve> you have the new config in PM
<ivve> blake_r: the thing is, i have an identical setup in a lab which works. i can provide that config also
<ivve> it drives me crazy that when we upgraded test, it worked. then went with prod, and now it just stopped working
<ivve> the setup is basically: only L3. so a maas vlan where dhcp happens and relays out to select vlans. the select vlans are the ones in the config, around 5-6 of them. all those vlans have helpers pointing to maas on its own vlan
<ivve> the lab has the exact same setup
<ivve> hmm when i removed everything and added it again i think the error disappeared when i pressed commission on a new node. waiting to see if pxe working
<ivve> got different error now
<ivve> PXE-E51: no dhcp or proxydhcp offers were received.
<ivve> seems like it doesn't offer the file on L3
<ivve> but it works on L2
<ivve> blake_r: is it possible to populate the database with existing deployed machines? im thinking of cleaning this out and repopulating it (not dumping the DB as i think the problem lies within it)
<Japje> for custom images (debian 10 in my case) is there a way to re-create the 90_dpkg* files so i can access the maas datasource?
<Japje> perhaps somehow from within curtin
<blake_r> ivve: are you sure there is not a duplicate DHCP server on the network?
<ivve> blake_r: very sure
<ivve> i can see the improper lease going out from rackd servers with a missing FNAME flags
<ivve> (with tcpdump)
<ivve> blake_r: im thinking something is wrong in the DB.. when looking at newly added nodes in the maasserver_node table bios_boot_method is empty, in comparison to the lab db it is populated with "pxe"
<ivve> i updated the row but no change
<blake_r> ivve: that is because the machine has never pxe booted from maas yet
<blake_r> ivve: you are still having the issue of getting the wrong fname
<blake_r> ivve: maas is not between the machine and the dhcp server
<blake_r> ivve: maas just configures isc-dhcp
<ivve> oh okay
<ivve> yes
<blake_r> ivve: something is wrong with the dhcp configuration
<ivve> BMC problems disappeared after reseting DHCP (removing and adding it)
<blake_r> ivve: but looking at the latest paste, it looks correct
<ivve> but it still missing FNAME
<mup> Bug #1839491 opened: Manully performed partitioning changes get reverted on reboot <MAAS:New> <https://launchpad.net/bugs/1839491>
<ivve> im a bit afraid of removing the vlan and subnet since i have lots of machines there
<blake_r> ivve: no you shouldn't need to do that
<blake_r> ivve: can you send me the output of dhcpdump
<ivve> but i disabled DHCP in the entire environment
<ivve> blake_r: coming up
<ivve> blake_r: so isc-dhcp-service is failed
<ivve> blake_r: i guess that is a problem?
<blake_r> ivve: yeah whats the reason for the failure?
<blake_r> ivve: wait no
<blake_r> ivve: you care about the maas-dhcpd service
<blake_r> ivve: not tha isc-dhcp-service
<blake_r> ivve: maas only controls the maas-dhcpd service
<ivve> that one is working and had it working entire time
<ivve> isc-dhcp-server.service  loaded failed failed ISC DHCP IPv4 server
<ivve> â isc-dhcp-server6.service loaded failed failed ISC DHCP IPv6 server
<ivve> disabled those and rebooted server, lets see
<ivve> in the lab those are disabled.. hmm
<ivve> blake_r: here is the bootpreply https://pastebin.com/raw/hp6VBZDY
<ivve> i guess the request isn't very interesting
<ivve> but it contains 67 (bootfile name)
<blake_r> ivve: i dont see option 67 in that pase
<blake_r> paste*
<ivve> blake_r: in the request?
<blake_r> ivve: that paste* seems to be the reply
<ivve> you want the request?
<ivve> https://pastebin.com/raw/NYx50kfA
<blake_r> ivve: the response should be getting you: filename "lpxelinux.0";
<ivve> blake_r: i know :)
<blake_r> ivve: can you check /var/log/syslog dhcpd logs in there
<ivve> blake_r: during a commission? nothing in there at all
<ivve> i've been checking /var/log/maas/*.log as well
<ivve> no errors pop or anything of particular interest
<ivve> so currently tailing /var/log/maas/*.log and syslog during a commission
<blake_r> ivve: is it showing the DHCP messages?
<ivve> nopes
<ivve> all empty
<ivve> i will check the lab environment if its the same
<ivve> slow HP machine booting :P
<ivve> okay i can confirm that no output to logfiles happen before the actual pxelinux file is downloaded
<ivve> so nothing on a working system outputs any dhcp data
<ivve> first thing that happens is:
<ivve> 2019-08-08 15:59:02 provisioningserver.rackdservices.tftp: [info] lpxelinux.0 requested by 10.23.5.30
<ivve> and that is in /var/log/maas/rackd.log
<blake_r> hmm okay
<blake_r> anything in journalctl -u maas-dhcpd
<blake_r> sorry if I am all over the place, but not really seen this issue before
<blake_r> expecially when the dhcpd.conf is correct
<ivve> blake_r: no worries man, we are kinda baffled here as well. ready to abandon maas due this issue
<blake_r> just to be sure nothing crazy is going on can you do a "ps auxf | grep dhcpd"
<blake_r> make sure no extra dhcpd are running
<blake_r> well its not really MAAS that is causing the issue as much as its isc-dhcp as MAAS just configures that
<ivve> just the maas process
<ivve> and since i only using 1 in the HA setup the "offline" dhcp has no processes
<ivve> checking journalctl now
<ivve> journalctl is empty on the "offline" node
<ivve> but lots of data in the active, checking now
<blake_r> try turning off HA for dhcp
<blake_r> lets see if that fixes it
<ivve> yes it is off
<blake_r> I just tested mine and got
<blake_r> FNAME: lpxelinux.0.
<ivve> alas the "offline" node
<ivve> no dhcp process going on there
<ivve> and maas is suggesting to enable HA
<ivve> ok found some of the request/offers
<ivve> https://pastebin.com/raw/nc8GSDbb
<blake_r> ivve: okay so its making an offer
<blake_r> ivve: can you check the dhcpdump of that over? you still have that running?
<ivve> yes
<ivve> its in a previous paste
<ivve> lemme check
<blake_r> ah thought that was a new offer
<ivve> here is the offer https://pastebin.com/raw/hp6VBZDY
<ivve> they are all the same, timestamps don't match
<ivve> but they are 100% identical
<blake_r> this is a physical machine booting or a vm?
<ivve> physical machine comission
<ivve> no errors at all
<ivve> just simply offer with no fname
<ivve> even tho option 67 is requested from client
<ivve> well the option is there, but its empty
<ivve> just .
<blake_r> is this a layer2 connection between the rack controller and the physical machine? or is it using a DHCP relay?
<ivve> relay
<blake_r> did you configure that VLAN in maas to be relayed?
<ivve> yes
<ivve> i can show a picture of the configurations in lab and prod
<ivve> they are identical
<ivve> one works, other not
<blake_r> you sure the relays are configured the same?
<ivve> and i can also paste the helper configuration in the swtich, which is also identical
<blake_r> well if its identical
<ivve> you can see for yourself
<ivve>  :)
<blake_r> I believe you
<ivve> we are all baffled here on how it doesn't work
<blake_r> :-)
<blake_r> if you could provide the dhcpd.conf dhcpd-interfaces and helper configuration between both envs and I can compare
<blake_r> just to see if anything pops out at me
<ivve> sure
<ivve> https://pasteboard.co/IrLxpI8.png
<ivve> https://pasteboard.co/IrLxPlb.png
<blake_r>  ivve: so I am confused some because that dhcpd.conf you provided only had one vlan in there
<blake_r> ivve: but for dhcp relay you must have both
<ivve> there is also relay to LOM network
<ivve> i didn't include that
<ivve> but BMC works, i can turn on/off and check power status
<blake_r> yeah so I think that is the issue then
<blake_r> as I didn't see it in the config
<blake_r> https://bugs.launchpad.net/maas/+bug/1836276
<blake_r> is the issue you are hitting
<ivve> hmm not sure what you mean
<ivve> both what?
<blake_r> there is also relay to LOM network
<blake_r> i didn't include that
<ivve> yea getting the shot one sec
<blake_r> I think that bug above is your issue
<blake_r> is the MAAS version between env the same?
<blake_r> or is 1 MAAS 2.5 and the other is MAAS 2.6?
<ivve> https://pasteboard.co/IrLztLf.png
<ivve> both are 2.6.0
<ivve> i can run a dist-upgrade on both machines and they have the stable ppa
<ivve> and no updates are found
<blake_r> strange
<ivve> yes
<blake_r> then that would not be the issue
<ivve> :)
<blake_r> as that bug says its a 2.6 only issue
<blake_r> but your not having that issue in your staging
<ivve> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
<ivve> just tried on all 4 machines
<blake_r> can you provide me the full dhcpd.conf on production and then the full dhcpd.conf on staging?
<ivve> yes
<ivve> production https://pastebin.com/raw/MYRTQvU8
<ivve> uhm sorry that was lab
<ivve> so here is the prod for real this time https://pastebin.com/raw/eaKGuhnD
<ivve> observe the lab has now HA enabled
<ivve> and last config for prod was to disable HA
<blake_r> i think the issue is that the subnets being relayed do not have the bootloaders defined
<blake_r> but in the staging environment is showing the same behaviour
<blake_r> in the config, but is working
<blake_r> which seems very wierd that it works in one case but not the other
<ivve> yes and also it was working before in 2.4.2
<ivve> in prod
<ivve> we have a pretty large env installed as you can see
<ivve> and its like 1 year old
<blake_r> yeah as the bug states is a 2.6 only issue as we changed to using http instead of tftp for most of the boot process
<ivve> aye
<blake_r> ivve try this
<ivve> switches are identical
<ivve> no network has changed at least in prod
<ivve> even firmwares are the same
<blake_r> modify /usr/lib/python3/dist-packages/provisioningserver/templates/dhcp/dhcpd.conf.template
<blake_r> add to the top
<ivve> i could check if the templates are identical
<blake_r> https://paste.ubuntu.com/p/hqB6nMc3Ft/
<ivve> i was looking for those before
<ivve> but couldn't find
<blake_r> yeah I know
<ivve> i guess i could have asked you
<ivve> :)
<blake_r> try adding that to the top
<blake_r> then restart rackd
<ivve> checksummed the templates
<blake_r> then try to DHCP and see if the FNAME is actually set
<ivve> they are the same
<ivve> do i need to remove anything else?
<ivve> or just add it on the very top
<ivve> i will reboot the entire machine, just for safe :P
<ivve> they are virtual now so its quick
<ivve> ok testing a commission
<ivve> nopes
<ivve> no-go
<ivve> doesn't offer at all
<ivve> maybe i need to check some stuff
<ivve> perhaps disable all dhcp and enable it again
<ivve> disabled all and enabled those 3, 1 dhcp and 2 relays
<ivve> still no offers
<ivve> well maybe the conf was wrong or smth
<ivve> ok the template was wrong
<ivve> blake_r: can't find the problem but its in the } else statement at the bottom that you wanted me to add
<blake_r> ivve: paste it all at line 15 of the file and down
<blake_r> ivve: you need the option arch defined at the top of the file to come first
<blake_r> ivve: that is probably the issue
<ivve> https://pastebin.com/raw/1k3cE7Qv
<ivve> blake_r: thats just the top of the generated config
<blake_r> yeah you need to move it down below the PXEClient if statement
<ivve> check
<ivve> blake_r: some assistance, anything i can search for to find it?
<ivve> or just a paste with the full file you want me to test
<blake_r> if you remove what you added
<blake_r> it will be line 15
<ivve> after BOOTLOADERS?
<ivve> between bootloaders and subnet dhcp snippets?
<ivve>  {{dhcp_subnet['bootloader']}} \n          {{endif}}
<ivve> replace those two lines with your paste?
<blake_r> no
<ivve> after class "PXE" statement or replace it?
<blake_r> after it
<blake_r> above
<ivve> check
<blake_r> Define lease time globally (can be overriden globally or per subnet
<ivve> https://pastebin.com/raw/FkYaXFjX
<ivve> like that?
<ivve> didn't seem to work either
<ivve> or could you just supply a full template or should i use like 2.4.2 template?
<ivve> if thats the test
<ivve> reboot solved it
<ivve> testing commission
<ivve> it works now
<ivve> ipxe was the problem
<ivve> now the question remains
<ivve> why does it work in the lab
<ivve> and not in prod
<ivve> ?!
<blake_r> did you add that snippet in production?
<blake_r> and it still didn't work?
<ivve> your changed fixed it
<ivve> not using ipxe
<blake_r> you where getting FNAME: . before
<blake_r> that was not related to ipxe
<blake_r> i think the issue is that relayes are not getting any bootloader selections
<blake_r> that change will allow the relay subnets to fallback to the global defined bootloaders
<ivve> you sure?
<blake_r> yeah it should
<ivve> blake_r: well huge thanks for the help, how do we ensure this is fixed for future releases?
<ivve> should i bugreport?
<blake_r> I have that bug report
<blake_r> its that same bug
<blake_r> i will work on it and get it on 2.6.1
<blake_r> so on upgrade it will stay working for you all
<blake_r> sorry for the issues it caused you
<ivve> no worries, big thanks for the assistance!
<blake_r> np
<blake_r> glad I could help
<ivve> guys around here wanted to abandon all hope and use something else :)
<ivve> whatever that would be.... :P
<ivve> there is nothing else that is as good as maas imo
<sbeattie> ahhh, blackhat, for all your rigorous talks: https://twitter.com/veorq/status/1159559785068429312
#maas 2019-08-09
<Japje> for custom images (debian 10 in my case) is there a way to re-create the 90_dpkg* files so i can access the maas datasource?
<Japje> perhaps somehow from within curtin
<ivve> anyone know if its possible with some modification to _not_ use lwIP in pxelinux?
<ivve> via maas
<ivve> to be honest i think the shift made in 2.4 to 2.6 with pxe will follow a lot of problems since the issue isn't in maas but in drivers and pxe. example HP machines older than gen9 don't work
<ivve> nvm figured it out :)
#maas 2019-08-11
<mup> Bug #1839787 opened: Usabilla form obscured <MAAS:New> <https://launchpad.net/bugs/1839787>
<mup> Bug #1839788 opened: Websocket API: config.update should take list <MAAS:New> <https://launchpad.net/bugs/1839788>
#maas 2020-08-03
<mup> Bug #1889042 changed: DNS/Bind issue, DNS stops working every day <MAAS:Triaged> <https://launchpad.net/bugs/1889042>
<greenstatic> hello, does anybody know why maas creates DNS entries when a host is marked as a kvm host for each host network interface, i.e. <INTERFACE>.<VLAN>.<HOSTNAME>.example.com ?
#maas 2020-08-04
<mup> Bug #1890262 opened: Error message with a custom image URL doesn't clear <ui> <MAAS:New> <https://launchpad.net/bugs/1890262>
<mup> Bug #1890262 changed: Error message with a custom image URL doesn't clear <ui> <MAAS:New> <https://launchpad.net/bugs/1890262>
<mup> Bug #1890262 opened: Error message with a custom image URL doesn't clear <ui> <MAAS:New> <https://launchpad.net/bugs/1890262>
<mup> Bug #1890348 opened: [UI][Network] long fabric names bleed into subnet <ui> <MAAS:New> <https://launchpad.net/bugs/1890348>
#maas 2020-08-05
<mup> Bug #1890468 opened: not able to import new image after MAAS upgrade <MAAS:New> <https://launchpad.net/bugs/1890468>
<mup> Bug #1890468 changed: not able to import new image after MAAS upgrade <MAAS:New> <https://launchpad.net/bugs/1890468>
<mup> Bug #1890468 opened: not able to import new image after MAAS upgrade <MAAS:New> <https://launchpad.net/bugs/1890468>
#maas 2020-08-06
<mup> Bug #1890513 opened: MAAS reports incorrect per NUMA node memory <MAAS:New> <https://launchpad.net/bugs/1890513>
<mup> Bug #1890513 changed: MAAS reports incorrect per NUMA node memory <MAAS:New> <https://launchpad.net/bugs/1890513>
<mup> Bug #1890513 opened: MAAS reports incorrect per NUMA node memory <MAAS:New> <https://launchpad.net/bugs/1890513>
<mup> Bug #1890513 changed: MAAS reports incorrect per NUMA node memory <MAAS:New> <https://launchpad.net/bugs/1890513>
<mup> Bug #1890513 opened: MAAS reports incorrect per NUMA node memory <MAAS:New> <https://launchpad.net/bugs/1890513>
<mup> Bug #1890527 opened: After adding virsh machine with existing address, multiple machines share address info <MAAS:New> <https://launchpad.net/bugs/1890527>
<mup> Bug #1890527 changed: After adding virsh machine with existing address, multiple machines share address info <MAAS:New> <https://launchpad.net/bugs/1890527>
<mup> Bug #1890527 opened: After adding virsh machine with existing address, multiple machines share address info <MAAS:New> <https://launchpad.net/bugs/1890527>
<mup> Bug #1890527 changed: After adding virsh machine with existing address, multiple machines share address info <MAAS:New> <https://launchpad.net/bugs/1890527>
<mup> Bug #1890527 opened: After adding virsh machine with existing address, multiple machines share address info <MAAS:New> <https://launchpad.net/bugs/1890527>
<smallsam> hey all, just jumped on to ask if the team is aware of issues with some of the image mirrors:
<smallsam> # wget -v http://images.maas.io/ephemeral-v3/daily/bionic/amd64/20200804/ga-18.04/lowlatency/boot-kernel
<smallsam> (images.maas.io)|91.189.88.136|:80... connected.
<mup> Bug #1890581 opened: LXD KVM doesn't know about network connectivity <MAAS:New> <https://launchpad.net/bugs/1890581>
<mup> Bug #1890590 opened: LXD composed KVM machines have disconnected network configuration <MAAS:New> <https://launchpad.net/bugs/1890590>
#maas 2020-08-07
<mup> Bug #1890859 opened: maas-region-controller-2.9 depends on maas transitional package <MAAS:Triaged> <https://launchpad.net/bugs/1890859>
<mup> Bug #1890859 changed: maas-region-controller-2.9 depends on maas transitional package <MAAS:Triaged> <https://launchpad.net/bugs/1890859>
#maas 2020-08-08
<mup> Bug #1890859 opened: maas-region-controller-2.9 depends on maas transitional package <MAAS:Triaged> <https://launchpad.net/bugs/1890859>
<mup> Bug #1890859 changed: maas-region-controller-2.9 depends on maas transitional package <MAAS:Triaged> <https://launchpad.net/bugs/1890859>
<mup> Bug #1890859 opened: maas-region-controller-2.9 depends on maas transitional package <MAAS:Triaged> <https://launchpad.net/bugs/1890859>
