Giter Club home page Giter Club logo

Comments (3)

Nipunich avatar Nipunich commented on May 23, 2024

Following table shows the list of file integration use cases that needs to be verified and implemented.

File Integration Use Cases

Use Case Sub Use case Ballerina FTP Module / File Module Status
Reading a file from a specified location in file system
From Local location improve
From FTP location
From FTP location in passive mode
From SFTP location
From SFTP location in passive mode
From FTPS location in passive mode
Writing a file to a specified location in file system
To Local location improve
To FTP location
To FTP location in passive mode
To SFTP location
To SFTP location in passive mode
To FTPS location in passive mode
Moving a file without a transformation
Move a file/folder within the same server
Get file properties
Get size of a file
Get last modified time of a file
Split and merge files
Splits files into multiple chunks
Merges files into a single file
Append content to an existing file
Read specified lines of a file
Read specific lines between given line numbers from a file
Reads a specific line from a file
Processing archived files
Archive files or folder in the file system
List all files inside zip file
Decompresses a zip file in the file system
Check and Fetch files in the file system
Search for a file based on a given file pattern, directory pattern in a specified location
Checks the existence of a file in a specified location.
Fetch a file by a pattern
Get the content list of a given path
Copy files from one location to another
Copy particular files with given file pattern from files on the local physical file system or a file on an FTP server to a specified location
Create a file or folder
Create a file or folder on the local physical file system with or without content
Create a file or folder on a FTP server with or without content
Deleting a file or folder
Delete a file / folder on the local physical file system
Delete a file / folder on the FTP server
Sending a file
Send a file to a specified location on the local physical file system
Send a file to a specified location on a ftp server
Connect to a FTP server through a Proxy
Connect to a ftp server through a proxy

from ballerina-integrator.

Nipunich avatar Nipunich commented on May 23, 2024

I have drafted a sample program to implement the functionality of the file listener listening to a local folder. Here when a modification is done to the listening folder, the files are processed and move them to a specified location.

import ballerina/file;
import ballerina/log;
import ballerina/internal;
import ballerina/io;

listener file:Listener localFolder = new ({
    path: "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/fs",
    recursive: false
});

service fileSystem on localFolder {

    resource function onModify(file:FileEvent m) {

        error? e = localFolder.__start();
        log:printInfo("File created");

        string source = "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/fs";
        string fileNamePattern = "xml";

        internal:Path src = new(source);

        internal:Path[]|error fileList = src.list();

        if (fileList is internal:Path[]) {
            foreach (var file in fileList) {
                string ext = file.getExtension();

                if (ext == fileNamePattern) {
                    string name = file.getName();

                    string srcPath = "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/fs";
                    srcPath = srcPath + "/" + name;

                    string target = "/Users/nipuni/Documents/Ballerina_Integrator/file-integration/out";
                    target = target + "/" + name;

                    internal:Path srcP = new(srcPath);
                    internal:Path dest = new(target);

                    processFile(ext, srcPath);
                    log:printInfo("Processing finished");

                    error? er = srcP.moveTo(dest);
                    io:println(er);
                    log:printInfo("Operation successful");

                }
            }
        }
    }
}

// User can define the the processing ( Following is an example).
public function processFile(string ext, string source) {
    log:printInfo("Started processing the file ");

    if (ext == "xml") {
        log:printInfo("Reading xml file");
        io:ReadableByteChannel rbc = io:openReadableFile(source);
        io:println("Get byte channel");
        io:ReadableCharacterChannel rch = new(rbc, "UTF8");
        var result = rch.readXml();

        if (result is xml) {
            json j1 = result.toJSON({
            
            });
        }
    }

    if (ext == "csv") {
        log:printInfo("Reading csv file");
        io:ReadableCSVChannel rCsvChannel = io:openReadableCsvFile(source);

        while (rCsvChannel.hasNext()) {
            var records = rCsvChannel.getNext();

            if (records is string[]) {
                io:println(records);
            }
        }
    }
}

from ballerina-integrator.

Nipunich avatar Nipunich commented on May 23, 2024

Following util functions are implemented to process xml and json files.

// Copyright (c) 2019 WSO2 Inc. (http://www.wso2.org) All Rights Reserved.
//
// WSO2 Inc. licenses this file to you under the Apache License,
// Version 2.0 (the "License"); you may not use this file except
// in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.

import ballerina/log;
import ballerina/io;
import ballerina/config;
import ballerina/http;

// Util function to read a json file.
public function readJsonFile(io:ReadableByteChannel result) returns json|error {

    io:ReadableCharacterChannel? charChannelResult = getCharChannel(result);
    var resultJson = charChannelResult.readJson();

    if (resultJson is json) {
        io:println("File content: ", resultJson);
        return resultJson;
    } else {
        log:printError("An error occured.", err = resultJson);
        return resultJson;
    }
}

// Util function to read a xml file.
public function readXmlFile(io:ReadableByteChannel result) returns xml|error? {

    io:ReadableCharacterChannel? charChannelResult = getCharChannel(result);
    var resultXml = charChannelResult.readXml();

    if (resultXml is xml) {
        io:println("File content: ", resultXml);
        return resultXml;
    } else {
        log:printError("An error occured.", err = resultXml);
        return resultXml;
    }
}

// Util function to convert a byte channel to a character channel.
public function getCharChannel(io:ReadableByteChannel getResult) returns io:ReadableCharacterChannel? {

    io:ReadableCharacterChannel? charChannel = new io:ReadableCharacterChannel(getResult, "utf-8");

    if (charChannel is io:ReadableCharacterChannel) {
        return charChannel;
    } else {
        log:printError("An error occured.");
        return;
    }
}

from ballerina-integrator.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.