Live API calls in tests

[stephan-buckmaster]

When running this test:
rake TEST=test/services/toolbox/open_meteo_test.rb

I get this printed on the screen:

WARNING: live API call in test. USE: stub_get_response(:get_location, status: ___, response: _______) do; ...; end WARNING: live API call in test. USE: stub_get_response(:get_current_and_todays_weather, status: ___, response: _______) do; ...; end WARNING: live API call in test. USE: stub_get_response(:get_location, status: ___, response: _______) do; ...; end WARNING: live API call in test. USE: stub_get_response(:get_current_and_todays_weather, status: ___, response: _______) do; ...; end ..

Not sure what the best way to cover this functionality in tests is. When I inspect what is returned by the OpenData service, it is quite lengthy:

#<OpenData latitude=30.269146, longitude=-97.75338, generationtime_ms=0.1710653305053711, utc_offset_seconds=-18000, timezone="America/Chicago", timezone_abbreviation="CDT", elevation=157.0, current_units=#<OpenData time="iso8601", interval="seconds", weather_code="wmo code", temperature_2m="°F", apparent_temperature="°F", precipitation="inch", rain="inch", showers="inch", snowfall="inch", cloud_cover="%">, current=#<OpenData time="2024-08-08T18:30", interval=900, weather_code=0, temperature_2m=96.6, apparent_temperature=98.4, precipitation=0.0, rain=0.0, showers=0.0, snowfall=0.0, cloud_cover=0>, daily_units=#<OpenData time="iso8601", weather_code="wmo code", temperature_2m_max="°F", temperature_2m_min="°F", apparent_temperature_max="°F", apparent_temperature_min="°F", precipitation_sum="inch", rain_sum="inch", showers_sum="inch", snowfall_sum="inch", precipitation_probability_max="%">, daily=#<OpenData time=["2024-08-07", "2024-08-08"], weather_code=[95, 1], temperature_2m_max=[99.7, 98.7], temperature_2m_min=[76.5, 76.8], apparent_temperature_max=[110.2, 110.7], apparent_temperature_min=[84.8, 85.5], precipitation_sum=[0.0, 0.0], rain_sum=[0.0, 0.0], showers_sum=[0.0, 0.0], snowfall_sum=[0.0, 0.0], precipitation_probability_max=[0, 0]>>

Look into webmock? https://thoughtbot.com/blog/how-to-stub-external-services-in-tests

[krschacht]

Thanks for flagging this. Yea, I was debating about this one. I actually left it on on purpose, unsure if that was a good idea. :) The reason I left it in is that this happens to be a free API with no credentials required so hitting this API ends up executing a lot of code paths that don't otherwise get executed. We do have a stubbing library in the app and in most other situations we stub out the API calls.

The two downsides of leaving it in is that (1) it might make the test flaky because the API could go down—although this has never happened yet, and (2) it means we can't run the test suite offline. But it's been really rare for me to ever try doing that.

I think that if I am going to leave this in here, then I should wrap it in a block that suppresses the warning. A block like intentionally_calling_real_api do; ... end. I was sort of leaving it in as a half decision since I was a bit undecided. But it's worked well enough that I think I should suppress the warning since it looks like a mistake to leave it like this.

[krschacht]

I pushed a commit yesterday that now lets you intentionally call real APIs but you have to wrap it in a block. Those warnings are now gone. While I was at it I took care of a few other different warnings that have been showing when rails tests are run. Now I just get a nice row of green dots! :)

[stephan-buckmaster]

Ok, I don't know about this way of testing; I feel the API operators may be mislead about the usage of their system, and as a developer I don't expect tests to trigger network requests. Now I saw the code also provides for a mocking mechanism, mocked_response_get... and mocked_response_post... but they don't see to be used.

It does work and it is nice to have that clean line of dots.

[krschacht]

It's good to raise this. And I don't know that this is ideal, but I think it's okay because the test suite is probably only hitting the API fewer than 20 times per week in total. The mocking of responses is used in all the other cases, but this one API is available publicly for free and doesn't require any API key to hit so I figured leaving this single one as a "hot test" is a nice way to run all the internal code inherent in API calls that is skilled whenever things are mocked out. It's probably not the right long term solution but I figured it was okay to leave in place for now.