Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Review API documentation on guides.rubygems.org #292

Open
maras1 opened this issue Feb 26, 2019 · 8 comments
Open

Review API documentation on guides.rubygems.org #292

maras1 opened this issue Feb 26, 2019 · 8 comments

Comments

@maras1
Copy link

maras1 commented Feb 26, 2019

Examining problems in Artifactory local repo for gems we discovered that bundle installs send wrong api calls to repository
If you execute bundle install the following wrong api cals are sent.

  1. Get https://rubygems.org/versions with incorrect syntax and without gem name. The corect api call should be
    Get https://rubygems.org/api/v1/versions/rails.json
  2. GET https://rubygems.org/api/v1/dependencies with incorrect syntax and without gem. The correct rest apo call should be:
    GET https://rubygems.org/api/v1/dependencies?gems=chef-vault
    below you will find artifactory apache log file with incorrect calls

10.151.80.71 - - [26/Feb/2019:08:45:15 +0100] "GET /artifactory/api/gems/rubygems/versions HTTP/1.1" 404 129 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies HTTP/1.1" 200 - "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=chef-vault HTTP/1.1" 200 3420 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=chef HTTP/1.1" 200 358751 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=addressable%2Cbundler%2Cbunny%2Cchef-config%2Cchef-zero%2Cdiff-lcs%2Cerubis%2Cffi%2Cffi-yajl%2Chighline%2Ciniparse%2Ciso8601%2Cjson%2Cmime-types%2Cmixlib-archive%2Cmixlib-authentication%2Cmixlib-cli%2Cmixlib-config%2Cmixlib-log%2Cmixlib-shellout%2Cnet-sftp%2Cnet-ssh%2Cnet-ssh-multi%2Cohai%2Cplist%2Cproxifier%2Cpry%2Crest-client%2Crspec-core%2Crspec-expectations%2Crspec-mocks%2Crspec_junit_formatter%2Cserverspec%2Cspecinfra%2Csyslog-logger%2Cuuidtools%2Cwin32-api%2Cwin32-certstore%2Cwin32-dir%2Cwin32-event%2Cwin32-eventlog%2Cwin32-mmap%2Cwin32-mutex%2Cwin32-process%2Cwin32-service%2Cwin32-taskscheduler%2Cwindows-api%2Cwindows-pr%2Cwmi-lite%2Cyajl-ruby HTTP/1.1" 200 406822 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=amqp%2Cextlib%2Cmoneta%2Cpuma%2Crdp-ruby-wmi%2Cruby-hmac%2Cruby-openid%2Cstomp%2Csystemu%2Ctreetop HTTP/1.1" 200 33745 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=abstract%2Camq-protocol%2Carchive-tar-minitar%2Cbcrypt_pbkdf%2Cbones%2Cbuilder%2Ccoderay%2Cechoe%2Cfuzzyurl%2Chashie%2Choe%2Chttp-accept%2Chttp-cookie%2Cipaddress%2Cjruby-pageant%2Claunchy%2Clibyajl2%2Cmethod_source%2Cmime-types-data%2Cmulti_json%2Cneedle%2Cnet-dhcp%2Cnet-scp%2Cnet-ssh-gateway%2Cnet-telnet%2Cnetrc%2Cnokogiri%2Cptools%2Cpublic_suffix%2Crack%2Crake%2Crbnacl%2Crbnacl-libsodium%2Crcov%2Crdoc%2Crspec%2Crspec-its%2Crspec-support%2Cruby_parser%2Csfl%2Cslop%2Csolve%2Cspoon%2Ctermios%2Ctest-unit%2Cthin%2Ctomlrb%2Cwin32-ipc%2Cwin32console%2Cyard HTTP/1.1" 200 156684 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=amq-client%2Cenglish%2Ceventmachine%2Cfacets%2Cffi-win32-extensions%2Cpolyglot%2Cruby-yadis%2Cstructured_warnings HTTP/1.1" 200 21917 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=ParseTree%2CRubyInline%2Callison%2Cbones-extras%2Cbones-git%2Ccamping%2Cconfiguration%2Ccucumber%2Cdaemons%2Cdep_selector%2Cdomain_name%2Ceventmachine-le%2Cfcgi%2Cgemcutter%2Chttp_parser.rb%2Clanguage%2Clittle-plugger%2Cloquacious%2Cmemcache-client%2Cmini_portile%2Cmini_portile2%2Cminitar%2Cminitar-cli%2Cminitest%2Cmolinillo%2Cmongrel%2Cpcap%2Cpkg-config%2Cpower_assert%2Cpreforker%2Cracc%2Crake-compiler%2Crexical%2Crubyforge%2Csemverse%2Csexp_processor%2Cspicycode-rcov%2Csqlite3%2Csyntax%2Ctenderlove-frex%2Ctest-spec%2Cweakling%2Cwin32-file HTTP/1.1" 200 98731 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=SexpProcessor%2CZenTest%2Cactiverecord%2Cactivesupport%2Cbones-rcov%2Cbones-rspec%2Cbones-rubyforge%2Cbones-zentest%2Ccgi_multipart_eof_fix%2Ccucumber-core%2Ccucumber-expressions%2Ccucumber-formatter-dots%2Ccucumber-wire%2Cdep-selector-libgecode%2Cevent-bus%2Cfastthread%2Cflexmock%2Cgem_plugin%2Cgherkin%2Cgherkin3%2Cgit%2Cjson_pure%2Cmab%2Cmarkaby%2Cmetaid%2Cmocha%2Cmulti_test%2Cpattern-match%2Cpowerbar%2Cterm-ansicolor%2Cunf%2Cwin32-file-stat HTTP/1.1" 200 183851 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=activemodel%2Cactiverecord-deprecated_finders%2Cansi%2Carel%2Cbackports%2Cc21e%2Cconcurrent-ruby%2Ccucumber-messages%2Ccucumber-tag_expressions%2Ci18n%2Cmetaclass%2Cos%2Cspruz%2Cthread_safe%2Ctins%2Ctrollop%2Ctzinfo%2Cunf_ext%2Czeitwerk HTTP/1.1" 200 73439 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=atomic%2Cbcrypt-ruby%2Ccurses%2Cfunctional-ruby%2Cgoogle-protobuf%2Cref HTTP/1.1" 200 19545 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=bcrypt HTTP/1.1" 200 1921 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:18 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=bundler HTTP/1.1" 200 14018 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
@segiddins
Copy link
Member

These are not incorrect calls, a gem repository should either return the appropriate response (see RubyGems.org for the canonical example), or a 404

@maras1
Copy link
Author

maras1 commented Feb 27, 2019

I rely on the information indicated in https://guides.rubygems.org/rubygems-org-api/

  1. The following methods for the version resource are described there:
    GET - /api/v1/versions/[GEM NAME].(json|yaml)
    GET - /api/v1/versions/[GEM NAME]/latest.json
    I do not see any Get / versions method there.
  2. The following methods for the dependencies resource are described there:
    GET - /api/v1/dependencies?gems=[COMMA DELIMITED GEM NAMES]
    IMHO sending rest api call without specifying a gem name is incorrect.

@segiddins
Copy link
Member

The compact index API might be undocumented on that page

@hsbt hsbt transferred this issue from rubygems/bundler Mar 14, 2020
@deivid-rodriguez deivid-rodriguez changed the title bundle install send wrong rest api calls to repository Review API documentation on guides.rubygems.org Sep 3, 2020
@deivid-rodriguez
Copy link
Member

Updated the title to reflect what would need to be done here.

@deivid-rodriguez deivid-rodriguez transferred this issue from rubygems/rubygems Jun 27, 2021
@decoyjoe
Copy link

The Compact Index API, as implemented in rubygems/compact_index, is still undocumented on https://guides.rubygems.org/rubygems-org-api/

Now that the Dependency API is going away 2023-05-10, the Compact Index API should be documented.

@coldnebo
Copy link

coldnebo commented May 31, 2023 •
edited

I'm not sure I understand the discussion here. We are running the bundler 2.4.13 and it is generating these calls and it takes 10 minutes to install or even check deps. This is a major performance impact. Are you saying Artifactory is wrong, or bundler is wrong? The fallback isn't falling back to the compact api properly. Is there a workaround?

$ bundle install --verbose
Running `bundle install --verbose` with bundler 2.4.13
Found changes from the lockfile, re-resolving dependencies because the dependencies in your gemfile changed
HTTP GET [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/versions)
HTTP 404 Not Found [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/versions)
Bundler::Fetcher::FallbackError: Net::HTTPNotFound: [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/versions)
/DONOTDELETE/shared/rvm/gems/ruby-3.1.2@app/gems/bundler-2.4.12/lib/bundler/fetcher/downloader.rb:45:in `fetch'
  /DONOTDELETE/shared/rvm/gems/ruby-3.1.2@app/gems/bundler-2.4.12/lib/bundler/fetcher/compact_index.rb:124:in `call'
...
HTTP GET [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/api/v1/dependencies)
HTTP 404 Not Found [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/api/v1/dependencies)
...
Fetching source index from [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/)
HTTP GET [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/quick/Marshal.4.8/amq-protocol-0.8.3.gemspec.rz)
HTTP 200 OK [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/quick/Marshal.4.8/amq-protocol-0.8.3.gemspec.rz)

@coldnebo coldnebo mentioned this issue Jun 1, 2023
Closed
@coldnebo
Copy link

coldnebo commented Jun 1, 2023

I'm moving my issue to #337 since I'm not sure it's related to the doc issue.

@coldnebo
Copy link

This is the issue that Artifactory opened with bundler. Their side of this issue is: https://jfrog.atlassian.net/browse/RTFACT-13635

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@coldnebo @segiddins @deivid-rodriguez @maras1 @decoyjoe