Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add python3 and virtual environment support #8

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
408845f
Rubocop fixes
mojibake-umd Jul 8, 2022
03b8f98
augeas_core fixture required for Puppet > 6
mojibake-umd Jul 8, 2022
369f5dc
remove jethrocarr/initfact dependency
mojibake-umd Jul 11, 2022
c10011b
pip install is no longer an option
mojibake-umd Jul 11, 2022
58c1b91
WIP: Manage python installation
mojibake-umd Jul 11, 2022
9a86749
Implement virtual environments
mojibake-umd Jul 12, 2022
9ad0a7f
Attempt to eliminate hardcoded paths for yas3fs and python
mojibake-umd Jul 14, 2022
4f15753
move simple defaults to init.pp
mojibake-umd Jul 15, 2022
c8ae014
Let python class or user manage pip
mojibake-umd Jul 15, 2022
904dee8
Remove python class
mojibake-umd Jul 15, 2022
95b6ff3
Rename package class to install class
mojibake-umd Jul 15, 2022
0c1c962
Add python_version specs
mojibake-umd Jul 15, 2022
65997fc
spec/defines/mount_spec.rb: Target Supported Operating Systems
mojibake-umd Jul 15, 2022
db2740f
Trigger install on vcsrepo refresh
mojibake-umd Jul 18, 2022
77dbb39
Need more specific pip version
mojibake-umd Jul 18, 2022
2aa7ef7
Fix install exec creates location
mojibake-umd Jul 26, 2022
58184fb
Optionally manage yas3fs requirements.txt
mojibake-umd Jul 27, 2022
2f51dae
default venv_path = /opt/yas3fs/venv
mojibake-umd Jul 28, 2022
0ff91d2
completely manage/unmanage python
Aug 26, 2022
dccddc5
Follow python class version scheme
Aug 26, 2022
d58d23c
pyenv needs major and minor version number
Aug 26, 2022
86829df
fix install creates cleanup on first vcs pull
Aug 26, 2022
7e4a1ec
Be more explicit with source command/built-in
Aug 26, 2022
bff20c6
pyvenv needs some base system pip packages
Aug 26, 2022
65061f6
Revert "pyvenv needs some base system pip packages"
Sep 6, 2022
2d0ada7
manage_python = false is better default
Sep 6, 2022
5a07cbc
params.pp: When to set manage_requirements = true
Sep 6, 2022
532c965
Install venv before yas3fs
Sep 6, 2022
87b2a8b
Virtual Environments are only validate with python3+
Sep 7, 2022
3cf8977
Make sure requirements installe for all OSes
Sep 8, 2022
963cafa
trigger requirements to be installed on change
Sep 9, 2022
3940ea6
Ensure venv parent dirs exist
Nov 4, 2022
2af2485
Add spec for venv_path deep path
Nov 4, 2022
c10fc2d
Version bump: 0.5.0
Nov 4, 2022
7451539
Finish /usr/local/bin support.
Nov 4, 2022
bf0b0fb
Avoid duplicate declarations.
Nov 11, 2022
aee8be0
Update documentation
Nov 14, 2022
8eeabe8
Move manage_requirements default
Nov 14, 2022
19b8dc8
Add some data types
Nov 14, 2022
0f984bc
Big version bump
Nov 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Attempt to eliminate hardcoded paths for yas3fs and python
Thank you @djtaylor
#5

Closes !5
mojibake-umd committed Jul 14, 2022
commit 9ad0a7f49cf87cff0e1d0315934c88481cf47bdd
7 changes: 5 additions & 2 deletions manifests/package.pp
Original file line number Diff line number Diff line change
@@ -61,9 +61,12 @@
$_exec_command = ". ${venv_path}/bin/activate && ${python_version} /var/tmp/yas3fs/setup.py install --prefix=${venv_path}"
$_exec_creates = "${venv_path}/bin/yas3fs"
}else{
$_exec_command = "/usr/bin/${python_version} /var/tmp/yas3fs/setup.py install"
$_exec_command = "/usr/bin/env ${python_version} /var/tmp/yas3fs/setup.py install"
# Users which install does not create /usr/bin/yasfs consider making a symlink to
# Wherever it got installed so puppet will not attempt to reinstall every run
# I can really only test on a Redhat machine and relay on rspec to test everything else
# - Ron (mojibake-umd) -
$_exec_creates = '/usr/bin/yas3fs'

}

exec { 'install yas3fs':
3 changes: 2 additions & 1 deletion spec/classes/yas3fs_spec.rb
Original file line number Diff line number Diff line change
@@ -39,7 +39,7 @@
}
it {
is_expected.to contain_exec('install yas3fs').with(
'command' => '/usr/bin/python3 /var/tmp/yas3fs/setup.py install',
'command' => '/usr/bin/env python3 /var/tmp/yas3fs/setup.py install',
'creates' => '/usr/bin/yas3fs',
'cwd' => '/var/tmp/yas3fs',
'require' => 'Vcsrepo[/var/tmp/yas3fs]',
@@ -89,6 +89,7 @@
'venv_path' => '/opt/yas3fs/venv'
}
end

it {
is_expected.to contain_exec('install yas3fs').with(
'command' => '. /opt/yas3fs/venv/bin/activate && python3 /var/tmp/yas3fs/setup.py install --prefix=/opt/yas3fs/venv',
13 changes: 6 additions & 7 deletions spec/defines/mount_spec.rb
Original file line number Diff line number Diff line change
@@ -30,7 +30,7 @@
'ensure' => 'present',
'path' => '/etc/init/s3fs-test-mount.conf',
'notify' => 'Service[s3fs-test-mount]',
'content' => /exec \/usr\/local\/bin\/yas3fs -f \"\$S3_URL\" \"\$LOCAL_PATH\"/
'content' => %r{exec /usr/bin/env yas3fs -f \"\$S3_URL\" \"\$LOCAL_PATH\"},
)
}

@@ -63,7 +63,7 @@
'ensure' => 'present',
'path' => '/etc/init/s3fs-test-mount.conf',
'notify' => 'Service[s3fs-test-mount]',
'content' => /exec \/opt\/yas3fs\/venv\/bin\/yas3fs -f \"\$S3_URL\" \"\$LOCAL_PATH\"/
'content' => %r{exec /opt/yas3fs/venv/bin/yas3fs -f \"\$S3_URL\" \"\$LOCAL_PATH\"},
)
}

@@ -103,7 +103,7 @@
'ensure' => 'present',
'path' => '/etc/systemd/system/s3fs-test-mount.service',
'notify' => 'Service[s3fs-test-mount]',
'content' => /ExecStart=\/usr\/bin\/yas3fs -f s3:\/\/test-bucket \/media\/test-mount/
'content' => %r{ExecStart=/usr/bin/env yas3fs -f s3://test-bucket /media/test-mount},
)
}

@@ -133,10 +133,9 @@

it {
is_expected.to contain_file('yas3fs-test-mount').with(
'content' => /\/opt\/yas3fs\/venv\/yas3fs -f s3:\/\/test-bucket \/media\/test-mount/
'content' => %r{/opt/yas3fs/venv/bin/yas3fs -f s3://test-bucket /media/test-mount},
)
}

end

context 'with sysvinit' do
@@ -159,7 +158,7 @@
'ensure' => 'present',
'path' => '/etc/init.d/s3fs-test-mount',
'notify' => 'Service[s3fs-test-mount]',
'content' => /PATH=\/usr\/local\/bin:\$PATH/
'content' => %r{PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:\$PATH},
)
}

@@ -188,7 +187,7 @@

it {
is_expected.to contain_file('yas3fs-test-mount').with(
'content' => /PATH=\/opt\/yas3fs\/venv:\/usr\/local\/bin:\$PATH/
'content' => %r{PATH=/opt/yas3fs/venv/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:\$PATH},
)
}
end
14 changes: 7 additions & 7 deletions templates/systemd.erb
Original file line number Diff line number Diff line change
@@ -13,17 +13,17 @@ Environment=AWS_ACCESS_KEY_ID=<%= @aws_access_key_id %> AWS_SECRET_ACCESS_KEY=<%
<% @options.each do |v| -%>
<% cmd_args = "#{cmd_args} --#{v}" -%>
<% end -%>
# TODO: systemd requires a fully qualified path... but Puppet doesn't give us a
# good way to query what the path to a particular binary is :-/ This may need
# some further work, PRs welcome
# Thank @djtaylor https://github.com/pcfens/puppet-yas3fs/pull/5
# /usr/bin/env yas3fs
<% if @venv_path -%>
WorkingDirectory=<%= @venv_path %>
Environment=PATH=<%= @venv_path %>/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/bin
ExecStart=<%= @venv_path %>/yas3fs -f <%= cmd_args %> <%= @s3_url %> <%= @local_path %>
Environment=PATH=<%= @venv_path %>/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ExecStart=<%= @venv_path %>/bin/yas3fs -f <%= cmd_args %> <%= @s3_url %> <%= @local_path %>
<% else %>
ExecStart=/usr/bin/yas3fs -f <%= cmd_args %> <%= @s3_url %> <%= @local_path %>
Environment=PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ExecStart=/usr/bin/env yas3fs -f <%= cmd_args %> <%= @s3_url %> <%= @local_path %>
<% end %>
ExecStop=/bin/umount <%= @local_path %>
ExecStop=/usr/bin/env umount <%= @local_path %>
RestartSec=5
Restart=on-failure
OOMScoreAdjust=-1000
4 changes: 2 additions & 2 deletions templates/sysvinit.erb
Original file line number Diff line number Diff line change
@@ -14,9 +14,9 @@
### END INIT INFO

<% if @venv_path -%>
PATH=<%= @venv_path %>:/usr/local/bin:$PATH
PATH=<%= @venv_path %>/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$PATH
<% else %>
PATH=/usr/local/bin:$PATH
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$PATH
<% end %>

export S3_URL="<%= @s3_url %>"
2 changes: 1 addition & 1 deletion templates/upstart.erb
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@ script
<% if @venv_path -%>
<%= "exec #{@venv_path}/bin/yas3fs -f #{cmd_args} \"$S3_URL\" \"$LOCAL_PATH\"" %>
<% else %>
<%= "exec /usr/local/bin/yas3fs -f #{cmd_args} \"$S3_URL\" \"$LOCAL_PATH\"" %>
<%= "exec /usr/bin/env yas3fs -f #{cmd_args} \"$S3_URL\" \"$LOCAL_PATH\"" %>
<% end %>
end script