MSR setup for http request (today's date)

[Talisker]

I would like to extract the daily gas price from my fuel supplier's API. The http call is as follows: https://octopus.energy/api/v1/tracker/G-1R-SILVER-2017-1-M/daily/past/90/1/

I need to pick the current (i.e. today's) gas price. If I search through the text manually I can find today's data, in this case 2022-07-20.

Can I set MSR up to do a http request and then extract the "unit_rate" into a variable so that I can set another rule up to determine if it would be cheaper to heat my hot water using the immersion heater or gas (FYI my off peak electricity price is £0.075 per kwh). Any help or guidance as always is much appreciated.

I'm running MSR on raspberry PI 4 in docker version 22168

Thanks.

{
		"date": "2022-07-20",
		"market_index": 62.8463,
		"cost": 15.094631164947945,
		"standing_charge": 15.0885,
		"unit_rate": 8.211,
		"usage": 0.0007467013698630137,
		"unit_charge": 0.0061311649479452055,
		"breakdown": {
			"unit_charge": {
				"Wholesale cost": 6.28463,
				"Environmental & social obligations": 0.0,
				"Delivery & networks": 0.9681,
				"100% green": 0.0,
				"Administration, financing & margin": 0.56727,
				"VAT": 0.391
			},`

[toggledbits]

The HTTP Request action can store the response into a variable (global or rule/local) that you can then use to access the data in other expressions. If the server declares that the response is JSON data, it will automatically be parsed to an object. From there, using your example, you can access the data in the object as usual. For example, in your example data, if we stored that in a variable called "lastquery",. you could access the "unit_rate" data in the response with lastquery.unit_rate.

[Talisker]

@toggledbits I am not sure if the returned result is JSON? I can obviously paste the response into a JSON formatter but I guess that is not the same thing? The result is unfortunately not just the 'latest' (today's rate), but a whole history of days which means I'd have to pick out the values under each day or today '2022-07-25'

https://octopus.energy/api/v1/tracker/G-1R-SILVER-2017-1-M/daily/past/90/1/?format=json

[toggledbits]

@talisker That's JSON. It's doable, but not trivial for the new players.

I would first set up a Rule to query the server each day at a particular time. That rule would use an HTTP Request action to store the response into a global variable (let's call it energy_data). You need to create the global variable first, just leave the expression blank.

Then make another global expression to fetch and hold the rate for today from the response by using the first statement to go through energy_data.periods until it finds the entry with date matching today's date... it would look something like this

first entry in energy_data.periods with entry.date == strftime( "%F" ): entry.unit_rate

(assuming unit_rate is the correct key/field in the entry; if not, just use the correct key name).

You can then create whatever other rule you need based on the rate fetched.

I tested this; works fine. Here's the rule that does the fetch every day at midnight:

And here is what the expressions look like:

You will also need to set http_request_action_maxresponse in the engine section of your reactor.yaml file to a value large enough to contain the result. I just set it to 256 (units are kilobytes). It appears that the structure of the URL may offer some control over the number of entries returned, so you might consider using that to reduce the overhead.

Edit: sure enough, if you change the "90" in the URL to a smaller number, it returns that many days of info, so use something like 7 and you'll reduce the query overhead considerably. You really only need today's anyway, right? (so maybe even 1 or 2 would work?)

[Talisker]

@toggledbits thank you once again! Very clear instructions. I am now up an running and have opened up new opportunites to extend my automation.

[Talisker]

I have made an http request and get the following JSON response which I have stored in a global variable gas_consumption (like the example above). See below.

I have then tried to the following expression to extract the consumption value:

Clearly the returned data "results" is not published for today (strftime( "%F) i.e. 4/9/2022.

How do I extract the gas consumption for a particular date?
How do I extract yesterday's gas consumption (in the case of the response below 3/9/2022)?

===group

{
	"count": 482,
	"next": "https://api.octopus.energy/v1/gas-meter-points/xxxxxxxxxxx/meters/xxxxxxxxxxx/consumption/?format=json&group_by=day&page=2&page_size=10",
	"previous": null,
	"results": [{
		"consumption": 0.0,
		"interval_start": "2022-09-03T00:00:00+01:00",
		"interval_end": "2022-09-03T01:00:00+01:00"
	}, {
		"consumption": 1.077,
		"interval_start": "2022-09-02T00:00:00+01:00",
		"interval_end": "2022-09-03T00:00:00+01:00"
	}, {
		"consumption": 0.669,
		"interval_start": "2022-09-01T00:00:00+01:00",
		"interval_end": "2022-09-02T00:00:00+01:00"
	}, {
		"consumption": 0.761,
		"interval_start": "2022-08-31T00:00:00+01:00",
		"interval_end": "2022-09-01T00:00:00+01:00"
	}, {
		"consumption": 0.614,
		"interval_start": "2022-08-30T01:00:00+01:00",
		"interval_end": "2022-08-31T00:00:00+01:00"
	}, {
		"consumption": 0.0,
		"interval_start": "2022-08-29T00:00:00+01:00",
		"interval_end": "2022-08-29T01:00:00+01:00"
	}, {
		"consumption": 0.569,
		"interval_start": "2022-08-28T00:00:00+01:00",
		"interval_end": "2022-08-29T00:00:00+01:00"
	}, {
		"consumption": 0.0,
		"interval_start": "2022-08-27T00:00:00+01:00",
		"interval_end": "2022-08-28T00:00:00+01:00"
	}, {
		"consumption": 0.455,
		"interval_start": "2022-08-26T00:00:00+01:00",
		"interval_end": "2022-08-27T00:00:00+01:00"
	}, {
		"consumption": 0.681,
		"interval_start": "2022-08-25T00:00:00+01:00",
		"interval_end": "2022-08-26T00:00:00+01:00"
	}]
}

===

[toggledbits]

You are on the right track. The problem with what you are doing is:

1. Your are comparing entry.date in the loop, but date is not a field in those structures. I believe you mean to use interval_start. Also, gas_consumption used to get the result is not the correct name/key either.
2. If you were to create a separate test variable using strftime("%F") as the entirety of its expression, I think you will quickly see the problem with the value being generated vs what you are trying to compare it to. I've done that below. See the problem? I'll leave it to you for the moment to come up with a fix.

[Talisker]

Background to the question
I've been experimenting a little and I need help. If I use the following expression:

I get the correct response (i.e. today's day number '16'th).

I can use this in a HTTP requst to download some tarrif information via an API from my energy supplier - i.e. today's gas unit rate tariff

What I'd like to do

I would like to download tomorrow's gas unit rate tariff. To do this I need to set up and expression with today's date + 1 day (in the case of my example expression above that would be '17' (today being the 16th).

Question: How do I take today's date and add 1 day - so that I can make the HTTP request for tomorrow's information?

Any guidance or suggestions would be appreciated.

[toggledbits]

Read the docs for Expressions (the Date/Time handling section in particular) and familiarize yourself with the tools available. I'll give you a couple of pointers:

The time() function can take arguments in several forms to build any reasonable date. The dateparts() function will return any timestamp or the current time/date as its components (hour, minute, year, month, day, etc., and better than using strftime() for the purpose).

Special hint: if you tell time() to make a timestamp for July 32, 2022, it will give you a timestamp for August 1, 2022 without complaint.

[Talisker]

@toggledbits Thanks for the prompts, following which I made some progress:

I have Test51 which contains tomorrow's date, but I am struggling to get the time in a sting representation. I am trying to represent the day in Test53 expression but I have a syntax error.

I would appreciate some more guidance.

[toggledbits]

THAT is the proper use of strftime(). The time is the (optional) second argument (i.e. you don't write Test53.strftime(...), you write strftime( "%F %T", Test53)). Again, read the docs. strftime() is a Reactor-specific function, documented lower down on the page.

Edit: specific correction for clarity.

[Talisker]

@toggledbits Thanks for your pointers. Partial success:

However, the expressions do not update! I use Tomorrow_Date in a Rule (HTTP Request):

https://api.octopus.energy/v1/products/SILVER-22-04-25/gas-tariffs/G-1R-SILVER-22-04-25-M/standard-unit-rates/?page_size=10&period_from=https://api.octohttps://api.octopus.energy/v1/products/SILVER-22-04-25/gas-tariffs/G-1R-SILVER-22-04-25-M/standard-unit-rates/?page_size=10&period_from=${{Tomorrow_Date}}&order_by=period

Is there a way to force an update on selected expressions, say at a set time every day?

[toggledbits]

Can you show your rule?

[Talisker]

@toggledbits

No Reset Reaction or Local Expressions are set.

The request URL is:
https://api.octopus.energy/v1/products/SILVER-22-04-25/gas-tariffs/G-1R-SILVER-22-04-25-M/standard-unit-rates/?page_size=10&period_from=https://api.octohttps://api.octopus.energy/v1/products/SILVER-22-04-25/gas-tariffs/G-1R-SILVER-22-04-25-M/standard-unit-rates/?page_size=10&period_from=${{Tomorrow_Date}}&order_by=period

[toggledbits]

OK. Move your global expressions into your rule so they are rule-based expressions. They will update each time the Interval triggers.