Comments (3)
Following table shows the list of file integration use cases that needs to be verified and implemented.
File Integration Use Cases
Use Case | Sub Use case | Ballerina FTP Module / File Module | Status |
---|---|---|---|
Reading a file from a specified location in file system | |||
From Local location | improve | ||
From FTP location | |||
From FTP location in passive mode | |||
From SFTP location | |||
From SFTP location in passive mode | |||
From FTPS location in passive mode | |||
Writing a file to a specified location in file system | |||
To Local location | improve | ||
To FTP location | |||
To FTP location in passive mode | |||
To SFTP location | |||
To SFTP location in passive mode | |||
To FTPS location in passive mode | |||
Moving a file without a transformation | |||
Move a file/folder within the same server | |||
Get file properties | |||
Get size of a file | |||
Get last modified time of a file | |||
Split and merge files | |||
Splits files into multiple chunks | |||
Merges files into a single file | |||
Append content to an existing file | |||
Read specified lines of a file | |||
Read specific lines between given line numbers from a file | |||
Reads a specific line from a file | |||
Processing archived files | |||
Archive files or folder in the file system | |||
List all files inside zip file | |||
Decompresses a zip file in the file system | |||
Check and Fetch files in the file system | |||
Search for a file based on a given file pattern, directory pattern in a specified location | |||
Checks the existence of a file in a specified location. | |||
Fetch a file by a pattern | |||
Get the content list of a given path | |||
Copy files from one location to another | |||
Copy particular files with given file pattern from files on the local physical file system or a file on an FTP server to a specified location | |||
Create a file or folder | |||
Create a file or folder on the local physical file system with or without content | |||
Create a file or folder on a FTP server with or without content | |||
Deleting a file or folder | |||
Delete a file / folder on the local physical file system | |||
Delete a file / folder on the FTP server | |||
Sending a file | |||
Send a file to a specified location on the local physical file system | |||
Send a file to a specified location on a ftp server | |||
Connect to a FTP server through a Proxy | |||
Connect to a ftp server through a proxy | |||
from ballerina-integrator.
I have drafted a sample program to implement the functionality of the file listener listening to a local folder. Here when a modification is done to the listening folder, the files are processed and move them to a specified location.
import ballerina/file;
import ballerina/log;
import ballerina/internal;
import ballerina/io;
listener file:Listener localFolder = new ({
path: "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/fs",
recursive: false
});
service fileSystem on localFolder {
resource function onModify(file:FileEvent m) {
error? e = localFolder.__start();
log:printInfo("File created");
string source = "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/fs";
string fileNamePattern = "xml";
internal:Path src = new(source);
internal:Path[]|error fileList = src.list();
if (fileList is internal:Path[]) {
foreach (var file in fileList) {
string ext = file.getExtension();
if (ext == fileNamePattern) {
string name = file.getName();
string srcPath = "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/fs";
srcPath = srcPath + "/" + name;
string target = "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/out";
target = target + "/" + name;
internal:Path srcP = new(srcPath);
internal:Path dest = new(target);
processFile(ext, srcPath);
log:printInfo("Processing finished");
error? er = srcP.moveTo(dest);
io:println(er);
log:printInfo("Operation successful");
}
}
}
}
}
// User can define the the processing ( Following is an example).
public function processFile(string ext, string source) {
log:printInfo("Started processing the file ");
if (ext == "xml") {
log:printInfo("Reading xml file");
io:ReadableByteChannel rbc = io:openReadableFile(source);
io:println("Get byte channel");
io:ReadableCharacterChannel rch = new(rbc, "UTF8");
var result = rch.readXml();
if (result is xml) {
json j1 = result.toJSON({
});
}
}
if (ext == "csv") {
log:printInfo("Reading csv file");
io:ReadableCSVChannel rCsvChannel = io:openReadableCsvFile(source);
while (rCsvChannel.hasNext()) {
var records = rCsvChannel.getNext();
if (records is string[]) {
io:println(records);
}
}
}
}
from ballerina-integrator.
Following util functions are implemented to process xml and json files.
// Copyright (c) 2019 WSO2 Inc. (http://www.wso2.org) All Rights Reserved.
//
// WSO2 Inc. licenses this file to you under the Apache License,
// Version 2.0 (the "License"); you may not use this file except
// in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
import ballerina/log;
import ballerina/io;
import ballerina/config;
import ballerina/http;
// Util function to read a json file.
public function readJsonFile(io:ReadableByteChannel result) returns json|error {
io:ReadableCharacterChannel? charChannelResult = getCharChannel(result);
var resultJson = charChannelResult.readJson();
if (resultJson is json) {
io:println("File content: ", resultJson);
return resultJson;
} else {
log:printError("An error occured.", err = resultJson);
return resultJson;
}
}
// Util function to read a xml file.
public function readXmlFile(io:ReadableByteChannel result) returns xml|error? {
io:ReadableCharacterChannel? charChannelResult = getCharChannel(result);
var resultXml = charChannelResult.readXml();
if (resultXml is xml) {
io:println("File content: ", resultXml);
return resultXml;
} else {
log:printError("An error occured.", err = resultXml);
return resultXml;
}
}
// Util function to convert a byte channel to a character channel.
public function getCharChannel(io:ReadableByteChannel getResult) returns io:ReadableCharacterChannel? {
io:ReadableCharacterChannel? charChannel = new io:ReadableCharacterChannel(getResult, "utf-8");
if (charChannel is io:ReadableCharacterChannel) {
return charChannel;
} else {
log:printError("An error occured.");
return;
}
}
from ballerina-integrator.
Related Issues (20)
- [SMB Connector]Unit tests need to be asserted.
- [VSCode][Windows] Cannot open projects on D: drive
- [JMS Connector]Unit tests need to be added
- [File Integration using Samba]Port should be corrected
- [File Integration Using Samba]Incorrect file name is printed
- [File Integration Using Samba]Error in getting the file size
- [Deploying on Docker][Deploying on Kubernetes][Docs] Mention to set the environmental variables
- [FTP Connector]It only outputs the file size when getting the file size but not the unit.
- [Docs] [Observability] Refactor and Improve Observability docs
- [Logging using Logstash and Kibana] nohup command does not work for windows
- [Docs] SaaS Integration Learn page throws "Page not found"
- [FTP Connector]Cannot read the file content when docker configuration is used
- [File Integration using FTP] Remove unnecessary comments from template
- Issues with Ballerina and Ballerina Plugin in VSCode HOT 1
- [GmailConnector] searchFilter 'query' not working with standard Gmail search query string
- Once a module is being built, it encourages the users to install Ballerina version 1.0.5
- EI Connectors redirect links broken HOT 1
- Module details are not available in Ballerina Central HOT 1
- [Doc] The copy to clipboard feature copies unwanted content
- [FTP Connector][ftp:ClientEndpointConfig]The list() method need to return more clear output
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ballerina-integrator.