Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu 14.04 LTS support #1651

Closed
3 tasks done
amercader opened this issue Apr 17, 2014 · 35 comments
Closed
3 tasks done

Ubuntu 14.04 LTS support #1651

amercader opened this issue Apr 17, 2014 · 35 comments
Labels
Milestone

Comments

@amercader
Copy link
Member

We should aim to support Ubuntu 14.04, probably by next release.

I'd expect the main issues would be on installation instructions and libraries / requirements compatibility.

Things to check:
http://packages.ubuntu.com/trusty

@amercader amercader added this to the CKAN 2.3 milestone Apr 17, 2014
@joetsoi
Copy link
Contributor

joetsoi commented Apr 17, 2014

Are we happy to build multiple packages for differing ubuntu versions? I think we can adapt the vagrant file from the datapackager branch of the ckan-packaging to build multiple debs quite easily.

@amercader
Copy link
Member Author

@joetsoi That sounds good, but then I think we need to run the tests on both platforms (Not sure if Travis will support that travis-ci/travis-ci#2046)

@amercader
Copy link
Member Author

Package install on 14.04 fails with:

vagrant@vagrant-ubuntu-trusty-64:~$ sudo dpkg -i python-ckan_2.2_amd64.deb
Selecting previously unselected package python-ckan.
(Reading database ... 63398 files and directories currently installed.)
Preparing to unpack python-ckan_2.2_amd64.deb ...
Unpacking python-ckan (2.2-1) ...
Setting up python-ckan (2.2-1) ...
Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 337, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 2279, in load_entry_point
    return ep.load()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 1989, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 24, in <module>
    from logging.config import fileConfig
  File "/usr/lib/python2.7/logging/config.py", line 29, in <module>
    import io
  File "/usr/lib/python2.7/io.py", line 51, in <module>
    import _io
ImportError: No module named _io

@nigelbabu
Copy link
Contributor

Package install will definitely fail because we're compiling things with the gcc version on 12.04 which is different from the version in 14.04.

@mav808
Copy link

mav808 commented Apr 21, 2014

FYI, changes with Apache defaults. Need to add the entry below or you'll get a permission denied, and also you should have .conf on the end of config files now so it should be ckan_default.conf.

Require all granted

Other than that I've managed to get the default ckan install from source working in 14.04.

@rparrapy
Copy link
Contributor

I just installed CKAN 2.2 from source following the docs.

As of today, solr-jetty can't be used because of this bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=740596

I ended up using solr-tomcat without problems.
Just wanted to let you know, in case it helps.

@Hoedic
Copy link

Hoedic commented May 1, 2014

I just installed CKAN 2.3a from sources.

  • Same as @marekbell for Apache2. The changes in the permission is documented here https://httpd.apache.org/docs/trunk/upgrading.html. It's more a question of Apache2 version: 2.2 works as previously, 2.4 needs Require all granted
  • Concerning solr-jetty bug, there is another workaround that using tomcat, which is declaring /usr/share/java/tomcat-coyote.jarin the jetty config file /etc/jetty/start.config

@nigelbabu nigelbabu self-assigned this Jun 12, 2014
ddohler added a commit to opendataphilly/opendataphilly-ckan that referenced this issue Aug 25, 2014
Largely copied from a Github gist (see comment in the playbook), this
provisions a Vagrant box containing a working installation of CKAN. CKAN
is installed from a release package, rather than from source; as best I
can tell we can extend it via extensions.

I tried to use Ubuntu 14.04 initially but the package install fails, and
it turns out that the current 2.2 release of CKAN doesn't support 14.04
(ckan/ckan#1651), so this uses Ubuntu 12.04.
ddohler added a commit to opendataphilly/opendataphilly-ckan that referenced this issue Aug 25, 2014
Largely copied from a Github gist (see comment in the playbook), this
provisions a Vagrant box containing a working installation of CKAN. CKAN
is installed from a release package, rather than from source; as best I
can tell we can extend it via extensions.

I tried to use Ubuntu 14.04 initially but the package install fails, and
it turns out that the current 2.2 release of CKAN doesn't support 14.04
(ckan/ckan#1651), so this uses Ubuntu 12.04.
@pwhipp
Copy link

pwhipp commented Sep 2, 2014

Thanks @Hoedic, your workaround did the trick for me.

@amercader
Copy link
Member Author

See also #1829

@seanh seanh assigned seanh and unassigned nigelbabu Oct 29, 2014
@seanh seanh added the 14.04 label Oct 29, 2014
@seanh
Copy link
Contributor

seanh commented Oct 29, 2014

Doing a source install of master on 14.04, the database and datastore setup seems to work fine and tests pass.

There are at least two problems with Solr though, I'm opening issues with label 14.04. I have a workaround for #2019 but not for #2018 yet.

seanh added a commit that referenced this issue Oct 29, 2014
User paster not ckan command, so it works for non-package installs as
well.
seanh added a commit that referenced this issue Oct 29, 2014
Fixes some failing tests on Ubuntu 14.04.

Fixes #2018.
seanh added a commit that referenced this issue Oct 29, 2014
Document workaround for an Ubuntu 14.04 bug with Jetty.

This fixes #2019 for source installs, but note that it'll have to be
fixed for the 14.04 package install as well.
seanh added a commit that referenced this issue Oct 29, 2014
It needs to be 0.0.0.0 not 127.0.0.1 to support running Solr on a
different machine.
seanh added a commit that referenced this issue Oct 30, 2014
Hoping this will fix the tests on Python 2.6
seanh added a commit that referenced this issue Oct 30, 2014
This default URL works with the default Solr setup and is needed for
most of CKAN to work at all.
@florianm
Copy link
Contributor

If that's any good, I got SolR to work with CKAN 2.3a (master) on Ubuntu 14.04 LTS by following digitalocean's instructions to "install SolR manually" with some CKAN-specific modifications.
The current solr-jetty package is broken and so outdated the Jetty docs are written on papyrus.

SolR

  • The solr archive has a working example solr server, complete with a jetty server
  • The SolR example /example is copied to /opt/solr
  • Jetty will run as a system service (upstart) by creating /etc/default/jetty
  • /opt/solr/etc/jetty-logging.xml is modified as per instructions
  • A solr system user is created, file system permissions are set
  • The jetty service is registered to start by default
  • The example apps "collection1" is renamed to "ckan"
mv /opt/solr/solr/collection1 /opt/solr/solr/ckan
sed -i "s/collection1/ckan/g" /opt/solr/solr/ckan/core.properties
rm -r /opt/solr/solr/data/*

# Link CKAN's solr schema.xml
mv /opt/solr/solr/ckan/conf/schema.xml /opt/solr/solr/ckan/conf/schema.bak
ln -s /mnt/ckan/default/src/ckan/ckan/config/solr/schema.xml /opt/solr/solr/ckan/conf/schema.xml

JTS for solr-spatial-field

If that worked, SolR will return an XML response to http://%(ckan.site_url)s:8983/solr/ckan/select/?fl=*,score&sort=score%20asc&q={!geofilt%20score=distance%20filter=true%20sfield=spatial_geom%20pt=42.56667,1.48333%20d=1}&fq=feature_code:PPL

Now, with the SolR core "collection1" renamed to "ckan", and the SolR admin GUI at %(ckan.site_url)s:8983/solr, the solr_url must include the core name, and, I cannot italicise this enough, must not have a trailing slash.

solr_url = http://127.0.0.1:8983/solr/ckan 

This works for one or several CKANs sharing the same SolR engine.

@joetsoi
Copy link
Contributor

joetsoi commented Oct 1, 2015

(note to self) really minor doc change needed

http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html#setup-solr

change line 15 and line 18 to line 16 and line 19
also see ckan/ckan-packaging#6 for changes needed for packaging

@mattfullerton
Copy link
Contributor

@frogsapo - if you are looking to deploy CKAN permanently from source, you should check out http://docs.ckan.org/en/latest/maintaining/installing/deployment.html

@patrickval
Copy link

Hello guys,

I am deploying CKAN 2.4 on Ubuntu 14.04, with jety 6 and SOLR 4.7.2, apache2 server and Postgres 9.3 in another machine.

After climbing the services have following error:

Debug false:

Server Error
An internal server error occurred

Debug true:

WebError Traceback:
⇝ SearchError: SOLR returned an error running query: {'sort': 'score desc, metadata_modified desc', 'fq': [u'owner_org:"73bffc86-3b1f-46c3-85d2-0fa0084ac53e" +site_id:"default" +state:active'], 'facet.mincount': 1, 'rows': 0, 'facet': u'false', 'facet.limit': '50', 'wt': 'json', 'q': ':', 'fl': 'id validated_data_dict'} Error: 'Not Found'

LOG:

Error - <class 'ckan.lib.search.common.SearchError'>: SOLR returned an error running query: {'sort': 'score desc, metadata_modified desc', 'fq': [u'owner_org:"73bffc86-3b1f-46c3-85d2-0fa0084ac53e" capacity:"publi$
URL: http://150.162.6.39:5000/
File '/usr/local/lib/python2.7/dist-packages/weberror/errormiddleware.py', line 162 in call
app_iter = self.application(environ, sr_checker)
File '/usr/local/lib/python2.7/dist-packages/webob/dec.py', line 147 in call
resp = self.call_func(req, _args, *_self.kwargs)
File '/usr/local/lib/python2.7/dist-packages/webob/dec.py', line 208 in call_func
return self.func(req, _args, *_kwargs)
File '/usr/local/lib/python2.7/dist-packages/fanstatic/publisher.py', line 234 in call
return request.get_response(self.app)
File '/usr/local/lib/python2.7/dist-packages/webob/request.py', line 1053 in get_response
application, catch_exc_info=False)
File '/usr/local/lib/python2.7/dist-packages/webob/request.py', line 1022 in call_application
app_iter = application(self.environ, start_response)
File '/usr/local/lib/python2.7/dist-packages/webob/dec.py', line 147 in call
resp = self.call_func(req, _args, *_self.kwargs)
File '/usr/local/lib/python2.7/dist-packages/webob/dec.py', line 208 in call_func
return self.func(req, _args, *_kwargs)
File '/usr/local/lib/python2.7/dist-packages/fanstatic/injector.py', line 54 in call
response = request.get_response(self.app)
File '/usr/local/lib/python2.7/dist-packages/webob/request.py', line 1053 in get_response
application, catch_exc_info=False)
File '/usr/local/lib/python2.7/dist-packages/webob/request.py', line 1022 in call_application
app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware.py', line 389 in inner
result = application(environ, start_response)
File '/usr/local/lib/python2.7/dist-packages/beaker/middleware.py', line 73 in call
return self.app(environ, start_response)
File '/usr/local/lib/python2.7/dist-packages/beaker/middleware.py', line 155 in call
return self.wrap_app(environ, session_start_response)
File '/usr/local/lib/python2.7/dist-packages/routes/middleware.py', line 131 in call
response = self.app(environ, start_response)
File '/usr/local/lib/python2.7/dist-packages/pylons/wsgiapp.py', line 125 in call
response = self.dispatch(controller, environ, start_response)
File '/usr/local/lib/python2.7/dist-packages/pylons/wsgiapp.py', line 324 in dispatch
return controller(environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 338 in call
res = WSGIController.call(self, environ, start_response)
File '/usr/local/lib/python2.7/dist-packages/pylons/controllers/core.py', line 221 in call
response = self._dispatch_call()
File '/usr/local/lib/python2.7/dist-packages/pylons/controllers/core.py', line 172 in _dispatch_call
response = self._inspect_call(func)
File '/usr/local/lib/python2.7/dist-packages/pylons/controllers/core.py', line 107 in _inspect_call
result = self._perform_call(func, args)
File '/usr/local/lib/python2.7/dist-packages/pylons/controllers/core.py', line 60 in _perform_call
return func(**args)
File '/usr/lib/ckan/default/src/ckan/ckan/controllers/home.py', line 182 in index
return base.render('home/index.html', cache_force=True)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 217 in render
loader_class=loader_class)
File '/usr/local/lib/python2.7/dist-packages/pylons/templating.py', line 249 in cached_template
return render_func()
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 155 in render_template
return render_jinja2(template_name, globs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 104 in render_jinja2
return template.render(**extra_vars)
File '/usr/local/lib/python2.7/dist-packages/jinja2/environment.py', line 894 in render
return self.environment.handle_exception(exc_info, True)
File '/usr/lib/ckan/default/src/ckan/ckan/templates/home/index.html', line 2 in top-level template code
{% set homepage_style = ( g.homepage_style or '1' ) %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/page.html', line 1 in top-level template code
{% extends "base.html" %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/base.html', line 103 in top-level template code
{%- block page %}{% endblock -%}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/page.html', line 19 in block "page"
{%- block content %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/home/index.html', line 14 in block "content"
{% block primary_content %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/home/index.html', line 15 in block "primary_content"
{% snippet "home/layout{0}.html".format(homepage_style) %}
File '/usr/lib/ckan/default/src/ckan/ckan/lib/jinja_extensions.py', line 255 in _call
return base.render_snippet(args[0], **kwargs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 79 in render_snippet
renderer='snippet')
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 217 in render
loader_class=loader_class)
File '/usr/local/lib/python2.7/dist-packages/pylons/templating.py', line 249 in cached_template
return render_func()
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 155 in render_template
return render_jinja2(template_name, globs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 104 in render_jinja2
return template.render(**extra_vars)
File '/usr/local/lib/python2.7/dist-packages/jinja2/environment.py', line 894 in render
return self.environment.handle_exception(exc_info, True)
File '/usr/lib/ckan/default/src/ckan/ckan/templates/home/layout1.html', line 31 in top-level template code
{% block featured_organization %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/home/layout1.html', line 32 in block "featured_organization"
{% snippet 'home/snippets/featured_organization.html' %}
File '/usr/lib/ckan/default/src/ckan/ckan/lib/jinja_extensions.py', line 255 in _call
return base.render_snippet(args[0], **kwargs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 79 in render_snippet
renderer='snippet')
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 217 in render
loader_class=loader_class)
File '/usr/local/lib/python2.7/dist-packages/pylons/templating.py', line 249 in cached_template
return render_func()
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 155 in render_template
return render_jinja2(template_name, globs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 104 in render_jinja2
return template.render(**extra_vars)
File '/usr/local/lib/python2.7/dist-packages/jinja2/environment.py', line 894 in render
return self.environment.handle_exception(exc_info, True)
File '/usr/lib/ckan/default/src/ckan/ckan/templates/home/snippets/featured_organization.html', line 1 in top-level template code
{% set organizations = h.get_featured_organizations() %}
File '/usr/lib/ckan/default/src/ckan/ckan/lib/helpers.py', line 1905 in get_featured_organizations
items=config_orgs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/helpers.py', line 1936 in featured_group_org
extras = logic.get_action(list_action)({}, {})
File '/usr/lib/ckan/default/src/ckan/ckan/logic/init.py', line 429 in wrapped
result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py', line 508 in organization_list
return _group_or_org_list(context, data_dict, is_org=True)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py', line 423 in _group_or_org_list
group_list.append(logic.get_action(action)(context, data_dict))
File '/usr/lib/ckan/default/src/ckan/ckan/logic/init.py', line 429 in wrapped
result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py', line 1232 in organization_show
return _group_or_org_show(context, data_dict, is_org=True)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py', line 1171 in _group_or_org_show
packages_field=packages_field)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/dictization/model_dictize.py', line 423 in group_dictize
group, just_the_count=True)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/dictization/model_dictize.py', line 414 in get_packages_for_this_group
q)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/init.py', line 429 in wrapped
result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/action/get.py', line 1771 in package_search
query.run(data_dict)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/search/query.py', line 364 in run
(query, e.reason))
SearchError: SOLR returned an error running query: {'sort': 'score desc, metadata_modified desc', 'fq': [u'owner_org:"73bffc86-3b1f-46c3-85d2-0fa0084ac53e" capacity:"public" +state:active +site_id:"default"'], 'f$

Anyone know how to solve?
Thank you.

@florianm
Copy link
Contributor

florianm commented Oct 7, 2015

Hi @patrickval,

  • tongue-in-cheek suggestion: use http://www.datacats.com - docker instead of source install
  • my personal opinion: use datacats
  • have you copied the ckan schema into the solr core? Your solr at least runs and works.
  • fwiw, I've written up my source install adventure incl. testing solr and making sure JTS is installed correctly.
  • If you want to go with datacats, I'm currently writing up my datacats install from barebone AWS EC2 VM to full stack here.

Lastly, before you try datacats, give them a week or two to fix datacats/datacats#334
HTH!

@mattfullerton
Copy link
Contributor

Could you consider integrating a Python 2.7.9+ interpreter into the python environment/package? This seems to be possible:
https://renoirboulanger.com/blog/2015/04/upgrade-python-2-7-9-ubuntu-14-04-lts-making-deb-package/

My reason: ckan/datapusher#83

That said, there would be an easier fix for that issue by supplying a different config.

@amercader
Copy link
Member Author

@mattfullerton See my thoughts on this here: ckan/datapusher#83 (comment)

@amercader amercader modified the milestones: CKAN 2.5, CKAN 2.4 (old) Oct 29, 2015
@davidread davidread added the WIP label Oct 29, 2015
amercader added a commit that referenced this issue Nov 6, 2015
Reviewed and updated docs for package install, source install and source
deployment.
amercader added a commit that referenced this issue Nov 6, 2015
amercader added a commit that referenced this issue Nov 10, 2015
When running on Python 2.6, httpretty doesn't play nice with Solr (it
causes read timeouts). The tests are still run but we avoid using Solr,
either by using `use_cache`=False on `package_show` or disabling the
automatic indexing of datatasets when running on Python 2.6.
amercader added a commit that referenced this issue Nov 10, 2015
@amercader
Copy link
Member Author

Fixed with #2726. Please test the new docs and packages and create new issues if there are still problems.

@dportnoy
Copy link

These instructions for install of Solr worked for me: https://www.digitalocean.com/community/tutorials/how-to-install-solr-5-2-1-on-ubuntu-14-04
(uses jetty)

@mrnugraha
Copy link

Hi all, i try to install CKAN on ubuntu 14.04 but still error when install solr-jetty always JSP support not configured.

@Conzar
Copy link
Contributor

Conzar commented Feb 15, 2016

@Mrobbin You can try using Puppet to install ckan as the puppet module supports Ubuntu 14.04 and handles installing solr.
https://forge.puppetlabs.com/landcareresearch/ckan

@mrnugraha
Copy link

@Conzar ok i want to try, thank you for your solution.

@lperepol
Copy link

lperepol commented Apr 1, 2016

This issue is marked as closed. There are still issues with installing solr.
Solr install is not working as documented in :
http://docs.ckan.org/en/latest/maintaining/installing/install-from-package.html

@mstantoncook
Copy link
Contributor

I agree with @lperepol. I think it's best to have an 14.04 alternative route/option in the docs that involves compiling jetty (3.6.2) from source. I'm happy to test/write the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests