Kettle [practice 02] TXT type files are classified and imported, execute SQL to convert data types and put them into storage (complete process instance cloud resource sharing: including sql+kjb+ktr+ t

The resource link is at the end of the article , Include files ：

1. Requirement specification

An application related to administrative division boundary data , Administrative division data from Hydrographic micrograph Download from , You need to import this data into the database GreenPlum, And standardize the data format . There are two types of source data ：

• Area _ spot .txt（ Area name and center point coordinates ）
• Area _ Noodles .txt（ Area name and area boundary point coordinates ）

The data needs to be converted into two types ：

• spot POINT(longitude,latitude)
• polygon POLYGON( POINT, POINT, POINT, ... )

2. Process instance

2.1 Point and face file data warehousing

The data cleaning process of point and surface is the same , Here we use surface data to illustrate , First of all Filter all face data files under the source data folder ：

Parse the contents of the file , Here are some points to pay attention to ：

• File name field