Merge pull request #12 from fabriziopandini/0.7.2

fabriziopandini · web-flow · commit 7f752413468f · 2016-12-22T23:31:48.000+01:00
0.7.2
diff --git a/.gitignore b/.gitignore
@@ -17,4 +17,4 @@ Vagrantfile
 
 # Test
 provisioning
-declarative.yaml
+*.yaml
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -35,3 +35,14 @@ ansible_group_vars_path and ansible_host_vars_path are not supported anymore
 # 0.7.0 (November 02, 2016)
 
 * introduced support for declarative cluster definition
+
+# 0.7.1 (November 04, 2016)
+
+* Minor fix
+
+# 0.7.2 (November 16, 2016)
+
+* issues #11 Allow management of Ansible vars for all hosts
+* issues #9 Create group_vars and host_vars directory only if necessary
+
+* breacking change: custom group of groups all_groups:children removed. Insteal use all (automatically created by ansible)
diff --git a/doc/declarative.md b/doc/declarative.md
@@ -295,9 +295,9 @@ kubernetes:
     ...
   ...
   ansible_group_vars:
-    etcd :
+    etcd:
       var1: ...
-    docker :
+    docker:
       var2: ...
       var3: ...
   ...
@@ -308,6 +308,17 @@ Group vars can be set to literal value or to Jinja2 value generators, that will
 - **context_vars** see below
 - **nodes**, list of nodes in the ansible_group to which the group_vars belong
 
+Additionally it is possible to set variables for all groups/all hosts, by setting vars for the pre-defined `all` group of groups:
+
+```yaml
+kubernetes:
+  ...
+  ansible_group_vars:
+    all:
+      var1: ...
+  ...
+```
+
 Ansible group vars will be stored into yaml files saved into `{cluster.ansible_playbook_path}\group_vars` folder.
 
 The variable  `cluster.ansible_playbook_path` defaults to the current directory  (the directory of the Vagrantfile)  +  `/provisioning`; this value can be changed like any other cluster attributes (see Defining cluster & cluster attributes).
@@ -384,18 +395,42 @@ Context vars can be set to literal value or to Jinja2 value generators, that wil
 
 - nodes, list of nodes in the ansible_group to which the group_vars belong
 
-
 > Context_vars generator are always executed before group_vars and host_vars generators; the resulting context, is given in input to group_vars and host_vars generators.
 
+> In addition to context vars for groups, it is possible to create context_vars for all groups/all hosts, by setting vars for the pre-defined `all` group of groups; in this case, intuitively, the list of nodes whitin the context contains all the nodes.
+
 Then, you can use the above context var when generating group_vars for host vars.
 
+```yaml
+kubernetes:
+  box: centos/7
+  master:
+    ...
+    ansible_groups:
+      - kb8-master
+  minions:
+    ...
+    ansible_groups:
+      - kb8-minions
+
+  ansible_context_vars:
+    all:
+      var0: "{{ nodes | count }}"
+    kb8-master:
+      var1: "{{ nodes | count }}"
+
+  ansible_group_vars:
+    all:
+      var0_from_context: "{{ context['var0'] }}"
+    kb8-master:
+      var1_from_context: "{{ context['var1'] }}"
+```
+
 ### Group of groups
 
 A useful ansible inventory feature is [group of groups](http://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups).
 
-By default vagrant-compose will generate a group named `[all_groups:children]` with all the ansible_groups defined in cluster configuration; however:
-- you cannot rename all_groups
-- you cannot exclude any ansible group from all_groups.
+By default ansible has a group named `[all]` with all nodes in the cluster.
 
 If you need higher control on groups of groups you can simply add a new item to the variable `config.cluster.ansible_groups` before creating nodes.
 
diff --git a/doc/programmatic.md b/doc/programmatic.md
@@ -302,9 +302,9 @@ end
 
 As you can see, `consul` and `docker` ansible_groups now include both nodes from `consul-agent` and `load-balancer` node set; vice versa, other groups like `registrator`, `consul-template`, `nginx` contain node only from one of the two nodes set.
 
-Ansible playbook can leverage on groups for providing machines with the required software stacks.
+Ansible playbook can additionally leverage on groups for providing machines with the required software stacks.
 
-NB. you can see resulting ansible_groups by using `debug` command with `verbose` equal to `true`.
+> NB. you can see resulting ansible_groups by using `debug` command with `verbose` equal to `true`.
 
 ### Defining group vars
 
@@ -329,6 +329,18 @@ config.cluster.compose('test') do |c|
 end
 ```
 
+Additionally, it is possible to set variables for all groups/all hosts, by setting vars for the pre-defined `all` group of groups:
+
+``` ruby
+config.cluster.compose('test') do |c|
+  ...
+  c.ansible_group_vars['all'] = lambda { |context, nodes|
+    return { 'var0' => nodes.length }
+  }
+  ...
+end
+```
+
 Ansible group vars will be stored into yaml files saved into `{cluster.ansible_playbook_path}\group_vars` folder.
 
 The variable  `cluster.ansible_playbook_path` defaults to the current directory  (the directory of the Vagrantfile)  +  `/provisioning`; this value can be changed like any other cluster attributes (see Defining cluster attributes).
@@ -379,6 +391,8 @@ end
 
 > Context_vars generator are always executed before group_vars and host_vars generators; the resulting context, is given in input to group_vars and host_vars generators.
 
+> In addition to context vars for groups, it is possible to create context_vars for all groups/all hosts, by setting vars for the pre-defined `all` group of groups; in this case, intuitively, the list of nodes whitin the context contains all the nodes.
+
 Then, you can use the above context var when generating group_vars for nodes in the `consul-agent` group.
 
 ``` ruby
@@ -397,9 +411,7 @@ end
 ### Group of groups
 A useful ansible inventory feature is [group of groups](http://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups).
 
-By default vagrant-compose will generate a group named `[all_groups:children]` with all the ansible_groups defined in cluster configuration; however:
-- you cannot rename all_groups
-- you cannot exclude any ansible group from all_groups.
+By default ansible has a group named `[all]` with all the nodes defined in cluster configuration.
 
 If you need higher control on groups of groups you can simply add a new item to the variable `config.cluster.ansible_groups` before creating nodes.
 
diff --git a/lib/vagrant/compose/config.rb b/lib/vagrant/compose/config.rb
@@ -62,8 +62,6 @@ def filterInventory(inventory)
           end
         end
 
-        ansible_groups['all_groups:children'] = ansible_groups.keys
-
         return ansible_groups
       end
 
diff --git a/lib/vagrant/compose/declarative/cluster.rb b/lib/vagrant/compose/declarative/cluster.rb
@@ -24,27 +24,27 @@ class Cluster
         # It defaults to current directory/provisioning
         attr_accessor :ansible_playbook_path
 
-      	# Costruttore di una istanza di cluster.
+        # Costruttore di una istanza di cluster.
         def initialize()
-        	@multimachine_filter = ""
+          @multimachine_filter = ""
         end
 
         # Implements cluster creation from a playbook file
-	      def from (file)
-	        # calls vagrant-playbook utility for executing the playbook file.
-	        playbook = YAML.load(pycompose (file))
+        def from (file)
+          # calls vagrant-playbook utility for executing the playbook file.
+          playbook = YAML.load(pycompose (file))
 
-	        # extract cluster attributes
-	        @name   								= playbook.keys[0]
-	        @box    								= playbook[@name]['box']
-	        @domain 								= playbook[@name]['domain']
-	        @ansible_playbook_path  = playbook[@name]['ansible_playbook_path']
+          # extract cluster attributes
+          @name                   = playbook.keys[0]
+          @box                    = playbook[@name]['box']
+          @domain                 = playbook[@name]['domain']
+          @ansible_playbook_path  = playbook[@name]['ansible_playbook_path']
 
           # extract nodes
           nodes = []
           playbook[@name]['nodes'].each do |node|
 
-          	boxname = node.keys[0]
+            boxname = node.keys[0]
 
             box             = node[boxname]['box']
             hostname        = node[boxname]['hostname']
@@ -64,74 +64,85 @@ def from (file)
           # extract ansible inventory, ansible_group_vars, ansible_host_vars
           ansible_groups = {}
           if playbook[@name].key?("ansible")
-          	ansible = playbook[@name]['ansible']
-
-          	# extract ansible inventory
-          	ansible_groups = ansible['inventory']
-
-          	# cleanup ansible_group_vars files
-	          # TODO: make safe
-	          ansible_group_vars_path = File.join(@ansible_playbook_path, 'group_vars')
-
-	          FileUtils.mkdir_p(ansible_group_vars_path) unless File.exists?(ansible_group_vars_path)
-	          Dir.foreach(ansible_group_vars_path) {|f| fn = File.join(ansible_group_vars_path, f); File.delete(fn) if f.end_with?(".yml")}
-
-	          #generazione ansible_group_vars file (NB. 1 group = 1 gruppo host ansible)
-	          ansible['group_vars'].each do |group, vars|
-	            # crea il file (se sono state generate delle variabili)
-	            unless vars.empty?
-	              # TODO: make safe
-	              File.open(File.join(ansible_group_vars_path,"#{group}.yml") , 'w+') do |file|
-	                file.puts YAML::dump(vars)
-	              end
-	            end
-	          end
-
-	          # cleanup ansible_host_vars files (NB. 1 nodo = 1 host)
-	          # TODO: make safe
-	          ansible_host_vars_path = File.join(@ansible_playbook_path, 'host_vars')
-
-	          FileUtils.mkdir_p(ansible_host_vars_path) unless File.exists?(ansible_host_vars_path)
-	          Dir.foreach(ansible_host_vars_path) {|f| fn = File.join(ansible_host_vars_path, f); File.delete(fn) if f.end_with?(".yml")}
-
-	          #generazione ansible_host_vars file 
-	          ansible['host_vars'].each do |host, vars|
-	            # crea il file (se sono state generate delle variabili)
-	            unless vars.empty?
-	              # TODO: make safe
-	              File.open(File.join(ansible_host_vars_path,"#{host}.yml") , 'w+') do |file|
-	                file.puts YAML::dump(vars)
-	              end
-	            end
-	          end
-          end
 
-	        return nodes, ansible_groups 
-	      end
+            ansible = playbook[@name]['ansible']
+
+            # extract ansible inventory
+            ansible_groups = ansible['inventory']
+
+            # cleanup ansible_group_vars files
+            # TODO: make safe
+            ansible_group_vars_path = File.join(@ansible_playbook_path, 'group_vars')
+
+            if File.exists?(ansible_group_vars_path)
+              Dir.foreach(ansible_group_vars_path) {|f| fn = File.join(ansible_group_vars_path, f); File.delete(fn) if f.end_with?(".yml")}
+            end
+
+            #generazione ansible_group_vars file (NB. 1 group = 1 gruppo host ansible)
+            if ansible.key?("group_vars")
+              ansible['group_vars'].each do |group, vars|
+                # crea il file (se sono state generate delle variabili)
+                unless vars.empty?
+                  FileUtils.mkdir_p(ansible_group_vars_path) unless File.exists?(ansible_group_vars_path)
+                  # TODO: make safe
+                  fileName = group.gsub(':', '_')
+                  File.open(File.join(ansible_group_vars_path,"#{fileName}.yml") , 'w+') do |file|
+                    file.puts YAML::dump(vars)
+                  end
+                end
+              end
+            end
+
+            # cleanup ansible_host_vars files (NB. 1 nodo = 1 host)
+            # TODO: make safe
+            ansible_host_vars_path = File.join(@ansible_playbook_path, 'host_vars')
+
+            if File.exists?(ansible_host_vars_path)
+              Dir.foreach(ansible_host_vars_path) {|f| fn = File.join(ansible_host_vars_path, f); File.delete(fn) if f.end_with?(".yml")}
+            end
+
+            #generazione ansible_host_vars file 
+            if ansible.key?("host_vars")
+              ansible['host_vars'].each do |host, vars|
+                # crea il file (se sono state generate delle variabili)
+                unless vars.empty?
+                  FileUtils.mkdir_p(ansible_host_vars_path) unless File.exists?(ansible_host_vars_path)
+                  
+                  # TODO: make safe
+                  File.open(File.join(ansible_host_vars_path,"#{host}.yml") , 'w+') do |file|
+                    file.puts YAML::dump(vars)
+                  end
+                end
+              end
+            end
+          end
 
+          return nodes, ansible_groups 
+        end
 
-	      # Executes pycompose command
-	      def pycompose (file)
-	        p_err = ""
-	        p_out = ""
 
-	        begin
-	          p_status = Open4::popen4("vagrant-playbook -f #{file}") do |pid, stdin, stdout, stderr|
-	              p_err = stderr.read.strip
-	              p_out = stdout.read.strip
-	          end
-	        rescue Errno::ENOENT
-	          raise VagrantPlugins::Compose::Errors::PyComposeMissing
-	        rescue Exception => e
-	          raise VagrantPlugins::Compose::Errors::PyComposeError, :message => e.message
-	        end
+        # Executes pycompose command
+        def pycompose (file)
+          p_err = ""
+          p_out = ""
+
+          begin
+            p_status = Open4::popen4("vagrant-playbook -f #{file}") do |pid, stdin, stdout, stderr|
+                p_err = stderr.read.strip
+                p_out = stdout.read.strip
+            end
+          rescue Errno::ENOENT
+            raise VagrantPlugins::Compose::Errors::PyComposeMissing
+          rescue Exception => e
+            raise VagrantPlugins::Compose::Errors::PyComposeError, :message => e.message
+          end
 
-	        if p_status.exitstatus != 0
-	          raise VagrantPlugins::Compose::Errors::PyComposeError, :message => p_err
-	        end
+          if p_status.exitstatus != 0
+            raise VagrantPlugins::Compose::Errors::PyComposeError, :message => p_err
+          end
 
-	        return p_out
-	      end
+          return p_out
+        end
 
       end
     end
diff --git a/lib/vagrant/compose/programmatic/cluster.rb b/lib/vagrant/compose/programmatic/cluster.rb
diff --git a/lib/vagrant/compose/version.rb b/lib/vagrant/compose/version.rb
