Knime: Accessing a REST API with dynamic query param
source link: https://blog.knoldus.com/knime-accessing-a-rest-api-with-dynamic-query-param/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Knime: Accessing a REST API with dynamic query param
Reading Time: 3 minutes
Nowadays Rest API is the most widely used way to share data, In which many API returns a subset of complete data in form of page.
Sometimes we need to append multiple query param in the URL to get some specific and filtered data.
In this blog, we will learn how to generate dynamic URLs by adding query param and get data.
Knime platform supports Rest interface with Get-Request and Post-Request Node.
Accessing Rest Interface in Knime
Below is the Knime Analytic workflow which include all the Knime Nodes required to access Rest API in this blog are as below:
Explaining the Configuration of each Node
Table Creator
This node is with default configuration, It read the table from the native Knime format, The Performance is
Counting Loop Start
The Loop Start is the node that starts a loop that is executed a predefined number of times. At the end of the loop you need LoopEnd, which collects the results from all loop iterations. All nodes in between are executed as many times as you specify in the dialog of LoopStart.
In this node configuration we give the number of time the loop will run.
Math Formula
This node evaluates a mathematical expression based on the values in a row or given expression. The computed results can be either appended as a new column or a variable.
In this node we configure the increment on the loop index, in this case, it is increasing by 1.
String Manipulation
This node Manipulates strings like Search and Replace, to Uppercase remove leading and trailing white spaces.
Here this node will append the page number to the fix part of URL
Get Request
This node is actually responsible for fetching the data from the Rest URL, Here we can hard code the URL if there is no dynamic query param.
Or we can take the URL from the variable that we created in the last node with query param as shown in the below screenshot.
Also, there are other different configurations we can do in our case it is not required.
Loop End
This node will be executed which will execute all the node inside Counting loop Start and Loop End, this we can leave with default configuration.
JSON Writer & MongoDB Writer
After we get the data from Rest API we can save it to disk or any database.
JSON Writer: Writes the JSON values to JSON file, In configuration just give folder location to write the file.
MongoDB Writer: The MongoDB Writer node inserts documents in JSON cells into a MongoDB database. It needs just the connection details.
Conclusion:
Source of data can be multiple, Knime support almost all source of data to be analysed and processed.
The above workflow was small example of reading data from rest API with data in multiple pages.
Recommend
-
22
> 之前休假了, 嗯, 我们继续 欢迎关注公众号 数据分析指北 数据分析指北 - KNIME 模块(Data Access类型模块之一IO模块...
-
41
有的同学最近两天打开 KNIME 时，也许发现了如下的更新提示： 4.2 升级提示 KNIME 在 7月 13日发布了 KNIME...
-
9
ICC Test Cricket Data Analysis using KNIME Reading Time: 4 minutes KNIME Analytics Platform is open-source software for creating data science applications and services. Intuitive, open, and cont...
-
2
Linear Regression with Knime Reading Time: 5 minutes Linear Regression is perhaps one of the most well kn...
-
6
Data Access with Knime Reading Time: 5 minutes One of the most common challenges faced by aspiring data scientists and dat...
-
6
Data Visualisation In KNIME Reading Time: 3 minutes KNIME is definitely a dream for data scientists. It makes the work of...
-
2
Reading Time: 3 minutes In this blog, we are going to see, what is the Knime analytics platform and its important features to create an analytics workflow in an easy way. Introduction to Knime Analytics Platform KNI...
-
5
KNIME for Supply Chain Management Reading Time: 7 minutes Supply Chain Management or SCM...
-
3
Accessing Postgres via REST using pRest Sep 3 ・5 min read
-
1
In HTTP request, same query parameter with multiple values are allowed. For example, for query parameter brand, the query string looks like this: brand=foo&brand=bar. How to parse value of this parameter to a list in Fast...
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK