=== aaa is now known as Guest6095
meenamaybe this should've been a bug report https://github.com/canonical/cloud-init/commit/0755cff078d5931e1d8e151bdcb84afb92bc0f02#commitcomment-12157782009:00
-ubottu:#cloud-init- Commit 0755cff in canonical/cloud-init "cc_final_message: don't create directories when writing boot-finished (#445)"09:00
=== lesbraz is now known as sbtaz
=== sbtaz is now known as sbraz
esvhey folks, I am a bit puzzled by the behavior of my azure instance, I push this cloud-init file to the machine: https://paste.centos.org/view/f640db8d before I encrypt the box, so far so good, the machine is deployed, the cloud-init config happens and the encryption kicks in, up until this point things work the way they are supposed to19:21
=== Sammitch_ is now known as Sammitch
esvhowever, if I fully stop the machine (deallocate), when the machine starts again, it recreates the testvg-testlv entry in /etc/fstab. isn't this part of the config supposed to happen only once? or the mount part is per-boot......mmmmm, I can check that.19:22
esvwell, documentation says once per instance for the mounts section19:25
esv.... interesting19:25
minimalesv: what do you mean *recreates*? does the cloud-init.log show it running twice?19:26
minimalesv: also when you restart the machine does it get the *sam* instance id or a new instance id?19:28
esvazure disk encryption replaces the /dev/mapper/testvg-testlv /data with the /dev/mapper entry of the entry in /etc/crypttab, but after the machine comes up from unallocated state, the testvg-testlv entry is back in fstab19:29
esvI am pretty sure is the same id, as I deployed another image w/o encryption and has the same number of entries in /var/lib/cloud/instances and the 1st one is the same id and timestamp.19:31
esvI am reproducing the issue again19:31
minimalso what does /var/log/cloud-init.log show? (assuming debugging is enabled)19:31
minimalit will show what each cloud-init module is doing19:32
esvguess I need to check that too, let me kick another run and will dive there. 19:32
blackboxswesv: cloud-init analyze show  # will also show what modules ran in a slightly easier format to read. If a config module is skipped you'll see "->config-mounts previously run" or something19:33
esvwill check.19:33
blackboxswanalyze show shows you per-boot whether modules are activated  on that particular boot19:33
minimalI've got to get used to those commands, I'm old fashioned and used to trawling through logfiles lol19:34
blackboxswno prob: that said Azure shows a couple of extra boot "stages" in analyze output if I recall due to some of the early boot preprovisioning system environments.19:34
esvminimal, me too. 19:35
blackboxswminimal: old-fashined gets us to the details anyway... we need to improve that tool so it's formatted in a simple machine readable format too19:35
esvhow does "analyze show" determine a new boot, that's always puzzled me (always ~ 10-18 months maybe)19:37
minimalperhaps we need c-iGPT to tell us what's wrong? ;-)19:37
blackboxswheh, esv : cloud-init analyze show just parses /var/log/cloud-init.log (or any cloud-init log provided to it) looking for the anchor messages for known cloud-init boot stages of the format:  Cloud-init v. <VERSION> running 'init-local' at XXX.19:44
esvso, I ran : cloud-init analyze show | egrep -n "Boot|mount"19:44
esvhere are the outputs19:44
blackboxswit's a simple tool that collects start/end timing logs from within our very noisy log to capture time offsets for various modules and boot stages19:44
esvthis is the encrypted machine:  https://paste.centos.org/view/2038365619:44
blackboxswc-GPT can now mine those answers for the future :)19:45
esvhere is the non-encrypted: https://paste.centos.org/view/e5d6186719:46
esvas you can tell, upto Boot #4 they're the same19:46
blackboxswesv: so encrypted machine: something changed on boot record 16 that re-allowed cc_mounts to run. Likely an instance-id change.  `egrep -i 'Cloud-init|previous iid' /var/log/cloud-init.log` to see when instance id's changed19:47
blackboxswif any cloud changes instance-id in metadata cloud-init re-runs all modules as if it's a fresh (green-field) deployment19:48
esvbut on the non-encrypted one I see mount module running twice, on the last boot, I stopped the machine completely 19:48
esvblackboxsw, i get it... I love c-i, wish I knew of it much earlier. I did a lousy job trying to implement something a similar concept @ job-2 (much more limited version) 19:50
blackboxswwe get a lot of help making cloud-init better from the broad community adoption here from different distros and clouds... so many hands make light work (and give us a better perspective on what lousy stuff just doesn't fly in other distros/clouds etc)19:51
blackboxsw</end_opensource_party_line> :)19:52
esvI was getting too much info, I switched the command to this: egrep "Cloud-init.*running 'init-local|previous iid" /var/log/cloud-init.log20:02
esvhere the output: https://paste.centos.org/view/147521b520:02
esvI've deployed a new server from the same image, it has the same cloud-init beginning as the previous 2.20:08
blackboxswesv: not sure from the looks of things. 2 instance id changes which accounts for re-running mount  again in this cases. probably needs a bit more debugging/triage to see what cloud-init is doing across that specific reboot if instance-id is staying the same20:14

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!