Selfhosted

60093 readers

1151 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.
Promotion posts require your active participation in selfhosting or related communities, or the post will be removed. No more than 10% of your posts or comments may be self-promotional, or your post will be removed. F/LOSS Exception: If your post is about a project that is completely open source & can be self-hosted in full without payment, your post is exempt from this rule as long as you continue to engage in comments.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

How To Parse JSON Data To A Human Readable Format [SOLVED] (lemmy.world)

submitted 2 weeks ago* (last edited 1 week ago) by irmadlad@lemmy.world to c/selfhosted@lemmy.world

44 comments fedilink hide all child comments

So, yes this is selfhosting related. I am working on an n8n flow to pull in weather data so that I can have this data on a dashboard. I can't find any dockerized weather forecasting apps. Most of them connect to a personal weather station, which might be an option in the future. For the time being, this is a little project I'm working on.

Partial JSON snippet:

spoiler

0	
json	
cod	"200"
message	0
cnt	40
list	
0	
dt	1780693200
main	
temp	29.4
feels_like	29.23
temp_min	29.4
temp_max	29.68
pressure	1019
sea_level	1019
grnd_level	984
humidity	42
temp_kf	-0.28
weather

I would like to display something like this:

spoiler

Current temperature: 23.25 °C
Feels like: 24.09 °C
Low / High: 23.25 °C / 23.60 °C
Humidity: 94%
Atmospheric pressure: 1023 hPa (sea level: 1023 hPa, ground level: 988 hPa)
Temperature correction factor (temp_kf): -0.35

Weather: Light rain
Weather code: 500
Short condition: Rain
Icon: 10d

So, this is for you devs or coders out there. I can produce the JSON data. I'm just not sure how to parse it to something meaningful. I'm sure Python will have to be incorporated, but unsure of how to proceed.

Maybe someone could point me in a direction to tuts, articles, or your own experience. Sorry the JSON data doesn't format correctly. Lemmy formatting doesn't seem to allow that.

ETA: For anyone wandering across this, here is the solution: https://lemmy.world/post/47961985/24201369 thanks to @captcha_incorrect for his generosity and time.

you are viewing a single comment's thread
view the rest of the comments

[–] irmadlad@lemmy.world 2 points 2 weeks ago (1 children)

I can provide a sample tomorrow.

That would be swell, but I wouldn't make it a priority. Just whenever you get a round to it. I have been given a ton of source material to read. Stayed up a little too late last night trying to assimilate it all. LOL

[–] captcha_incorrect@lemmy.world 2 points 1 week ago* (last edited 1 week ago) (2 children)

Sharing a workflow requires a paid plan but could should be able to paste this in the editor to get the same effect.

I made two different sample flows, on that call the api and responds with directly with the response data (transformed). The other workflow is to same the data in a data table, and call the weather api on a schedule. You could even use ntfy and send notifications on a schedule if it will rain within an hour.

Edit the nodes to suite your need (URL, content of response etc).

n8n sample workflow (copy and paste in n8n's editor)

{
  "nodes": [
    {
      "parameters": {
        "url": "https://opendata-download-metfcst.smhi.se/api/category/snow1g/version/1/geotype/point/lon/16.158/lat/58.5812/data.json",
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        656,
        336
      ],
      "id": "d47a17c9-9bc7-448c-841a-f3b2d787b2cb",
      "name": "Sample weather JSON data"
    },
    {
      "parameters": {
        "jsCode": "let output = [];\n\nfor (const item of $input.first().json.timeSeries) {\n  output.push({\n    Time: item.time,\n    \"Current temperature\": item.data.air_temperature + ' °C'\n    // Add more as needed\n  });\n}\n\nreturn output;\n\n"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        880,
        240
      ],
      "id": "cf4f11a6-0ecc-4e96-afcb-a7903113da19",
      "name": "Convert json to array"
    },
    {
      "parameters": {
        "mode": "raw",
        "jsonOutput": "={\n  \"Time\": {{ $json.timeSeries[0].time }},\n  \"Current temperature\": {{ $json.timeSeries[0].data.air_temperature }},\n}\n",
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        880,
        432
      ],
      "id": "2f3765ba-6ee5-44b7-ac45-c5a407ed43e9",
      "name": "Manual mapping of one item",
      "disabled": true
    },
    {
      "parameters": {
        "options": {}
      },
      "type": "n8n-nodes-base.respondToWebhook",
      "typeVersion": 1.5,
      "position": [
        1104,
        240
      ],
      "id": "badd76a3-2920-4592-baa6-5202b4badba0",
      "name": "Respond to Webhook"
    },
    {
      "parameters": {
        "operation": "upsert",
        "dataTableId": {
          "__rl": true,
          "mode": "list",
          "value": ""
        },
        "columns": {
          "mappingMode": "defineBelow",
          "value": {},
          "matchingColumns": [],
          "schema": [],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {}
      },
      "type": "n8n-nodes-base.dataTable",
      "typeVersion": 1.1,
      "position": [
        1104,
        16
      ],
      "id": "c76d9dfa-9ec1-421f-a96b-d1a65efb0dd7",
      "name": "insert (or update if exists) table with data"
    },
    {
      "parameters": {
        "content": "Create the table first, then edit and select how to map the fields\n\n",
        "height": 224,
        "width": 320
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        992,
        -48
      ],
      "typeVersion": 1,
      "id": "51347b11-fcb8-49db-bd17-405ec8719ec0",
      "name": "Sticky Note"
    },
    {
      "parameters": {
        "jsCode": "let output = [];\n\nfor (const item of $input.first().json.timeSeries) {\n  output.push({\n    Time: item.time,\n    \"Current temperature\": item.data.air_temperature + ' °C'\n    // Add more as needed\n  });\n}\n\nreturn output;\n\n"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        880,
        16
      ],
      "id": "3967c4c5-5443-40d2-b1fe-830aa9327118",
      "name": "Convert json to array1"
    },
    {
      "parameters": {
        "url": "https://opendata-download-metfcst.smhi.se/api/category/snow1g/version/1/geotype/point/lon/16.158/lat/58.5812/data.json",
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        656,
        16
      ],
      "id": "f0043f80-c940-43cf-a5ef-2f7fe6dcc6a7",
      "name": "Sample weather JSON data1"
    },
    {
      "parameters": {
        "path": "1572b231-edc5-4516-935c-f205b737c5d2",
        "responseMode": "responseNode",
        "options": {}
      },
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 2.1,
      "position": [
        432,
        336
      ],
      "id": "30953c4e-7627-4646-8b8d-ca46cc06c841",
      "name": "Webhook - Respond after API call",
      "webhookId": "1572b231-edc5-4516-935c-f205b737c5d2"
    },
    {
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "minutes"
            }
          ]
        }
      },
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.3,
      "position": [
        448,
        16
      ],
      "id": "e242dea7-dc3c-4c0a-9630-786bf2a39b8a",
      "name": "Schedule to get weather data"
    },
    {
      "parameters": {
        "path": "1572b231-edc5-4516-935c-f205b737c5d2",
        "responseMode": "responseNode",
        "options": {}
      },
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 2.1,
      "position": [
        448,
        -240
      ],
      "id": "1c1ac8d1-5f64-4bab-a3ac-20f223b1411e",
      "name": "Webhook - respond with table data",
      "webhookId": "1572b231-edc5-4516-935c-f205b737c5d2"
    },
    {
      "parameters": {
        "operation": "get",
        "dataTableId": {
          "__rl": true,
          "mode": "list",
          "value": ""
        }
      },
      "type": "n8n-nodes-base.dataTable",
      "typeVersion": 1.1,
      "position": [
        656,
        -240
      ],
      "id": "7433815b-229c-4ad8-88ce-2bcff688ece5",
      "name": "Get row(s)"
    },
    {
      "parameters": {
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        864,
        -240
      ],
      "id": "76cd6bfe-da95-41c0-a3ee-812956eab79f",
      "name": "Edit Fields"
    },
    {
      "parameters": {
        "options": {}
      },
      "type": "n8n-nodes-base.respondToWebhook",
      "typeVersion": 1.5,
      "position": [
        1072,
        -240
      ],
      "id": "3c83ebc3-2c09-4d80-8278-e325237ae0e2",
      "name": "Respond to Webhook1"
    },
    {
      "parameters": {
        "content": "No code behind these nodes, only a skeleton sample for the workflow could look like",
        "height": 208,
        "width": 944
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        384,
        -304
      ],
      "typeVersion": 1,
      "id": "db35642a-04dd-427b-9a21-d4ad64dd5522",
      "name": "Sticky Note1"
    },
    {
      "parameters": {
        "content": "## Via data tabel\n",
        "height": 592,
        "width": 1088,
        "color": 3
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        288,
        -384
      ],
      "typeVersion": 1,
      "id": "877a2308-c70f-4b9e-8ef7-5dea94a812b8",
      "name": "Sticky Note2"
    },
    {
      "parameters": {
        "content": "## Direct response from remote API",
        "height": 400,
        "width": 1088,
        "color": 5
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        288,
        224
      ],
      "typeVersion": 1,
      "id": "d06d7c02-a8eb-494b-8d10-142fb2c04037",
      "name": "Sticky Note3"
    }
  ],
  "connections": {
    "Sample weather JSON data": {
      "main": [
        [
          {
            "node": "Convert json to array",
            "type": "main",
            "index": 0
          },
          {
            "node": "Manual mapping of one item",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert json to array": {
      "main": [
        [
          {
            "node": "Respond to Webhook",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert json to array1": {
      "main": [
        [
          {
            "node": "insert (or update if exists) table with data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sample weather JSON data1": {
      "main": [
        [
          {
            "node": "Convert json to array1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Webhook - Respond after API call": {
      "main": [
        [
          {
            "node": "Sample weather JSON data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Schedule to get weather data": {
      "main": [
        [
          {
            "node": "Sample weather JSON data1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Webhook - respond with table data": {
      "main": [
        [
          {
            "node": "Get row(s)",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get row(s)": {
      "main": [
        [
          {
            "node": "Edit Fields",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Edit Fields": {
      "main": [
        [
          {
            "node": "Respond to Webhook1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {},
  "meta": {
    "instanceId": "139fc1c0283616d9fae3813879a161cf8e8e0992eea7a9e87dc980c7d4ed97ec"
  }
}

[–] irmadlad@lemmy.world 2 points 1 week ago* (last edited 1 week ago)

Oh Snap! You didn't. I will definitely get this running in a bit when I finish all my chores and can go play. Wow man, thank you so much!! I had put this on my dashboard as a means of encouragement to finish what I started.

spoiler

ETA: That's so purty. Haven't fired it up, just imported. Got chores to do. Will report back.

spoiler

[–] irmadlad@lemmy.world 2 points 1 week ago (1 children)

Just a follow up to first say thank you, and that the script is working for me. I'm doing a bit of tweaking but it certainly jump started the process. You're a scholar and a gentleman.

[–] captcha_incorrect@lemmy.world 2 points 1 week ago

Glad to hear that you got it working! The hard part for me was to figure out how to transform a JSON string into an array of n8n objects. I had already done that before so now I just copy that node whenever I need something like again.

I’m doing a bit of tweaking

I figured you would have. But that is half the fun. :-)