Feapder的管道扩展

Related tags

Miscellaneousfeapder_pipelines
Overview

FEAPDER 管道扩展

简介

此模块为feapder的pipelines扩展，感谢广大开发者对feapder的贡献

随着feapder支持的pipelines越来越多，为减少feapder的体积，特将pipelines提出，使用者可按需安装

管道

PostgreSQL

贡献者：沈瑞祥

联系方式：[email protected]

安装

pip install feapder_pipelines[pgsql]

使用

在feapder项目的setting.py中使用如下配置

# PostgreSQL
PGSQL_IP = 
PGSQL_PORT = 
PGSQL_DB = 
PGSQL_USER_NAME = 
PGSQL_USER_PASS = 

ITEM_PIPELINES = [
    "feapder_pipelines.pipelines.pgsql_pipeline.PgsqlPipeline"
]
细节 注：入库时 ON CONFLICT(key) 默认为id或通过如下sql查出来的第一个值 
select column_names from(
        select
            t.relname as table_name,
            i.relname as index_name,
            array_to_string(array_agg(a.attname), ', ') as column_names
        from
            pg_class t,
            pg_class i,
            pg_index ix,
            pg_attribute a
        where
            t.oid = ix.indrelid
            and i.oid = ix.indexrelid
            and a.attrelid = t.oid
            and a.attnum = ANY(ix.indkey)
            and t.relkind = 'r'
            and t.relname like '%'
        group by
            t.relname,
            i.relname
        order by
            t.relname,
            i.relname) as res
    where table_name = 'table_name';
You might also like...
Releases(v1.0.3)
Owner
boris
爬虫工程师、数据工程师
boris
GitHub Repository
A community-driven python bot that aims to be as simple as possible to serve humans with their everyday tasks

JARVIS on Messenger Just A Rather Very Intelligent System, now on Messenger! Messenger is now used by 1.2 billion people every month. With the launch

Swapnil Agarwal 1.3k Jan 07, 2023
Module to align code with thoughts of users and designers. Also magically handles navigation and permissions.

This readme will introduce you to Carteblanche and walk you through an example app, please refer to carteblanche-django-starter for the full example p

Eric Neuman 42 May 28, 2021
A collection of convenient parsers for Advent of Code problems.

Advent of Code Parsers A collection of convenient Python parsers for Advent of Code problems. Installation pip install aocp Quickstart You can import

Miguel Blanco Marcos 3 Dec 13, 2021
A faster copy of nell's comet nuker

Astro a faster copy of nell's comet nuker also nell uses external libraries like it's cocaine man never learned to use ansi color codes (ily nell) (On

horrid 8 Aug 15, 2022
A promo calculator for sports betting odds.

Sportbetter Calculation Toolkit Parlay Calculator This is a quick parlay calculator that considers some of the common promos offered. It is used to id

Luke Bhan 1 Sep 08, 2022
디텍션 유틸 모음

Object detection utils 유틸모음 설명 링크 convert convert 관련코드 https://github.com/AI-infinyx/ob_utils/tree/main/convert crawl 구글, 네이버, 빙 등 크롤링 관련 https://gith

codetest 41 Jan 22, 2021
Simple calculator made in python

calculator Uma alculadora simples feita em python CMD, PowerShell, Bash ✔️ Início 💻 apt-get update apt-get upgrade -y apt-get install python git git

Spyware 8 Dec 28, 2021
Script to calculate delegator epoch returns for all pillars

znn_delegator_calculator Script to calculate estimated delegator epoch returns for all Pillars, so you can delegate to the best one. You can find me o

2 Dec 03, 2021
Stock Monitoring

Stock Monitoring Description It is a stock monitoring script. This repository is still under developing. Getting Started Prerequisites & Installing pi

Sission 1 Feb 03, 2022
Python-geoarrow - Storing geometry data in Apache Arrow format

geoarrow Storing geometry data in Apache Arrow format Installation $ pip install

Joris Van den Bossche 11 Mar 03, 2022
Pokemon catch events project to demonstrate data pipeline on AWS

Pokemon Catches Data Pipeline This is a sample project to practice end-to-end data project; Terraform is used to deploy infrastructure; Kafka is the t

Vitor Carra 4 Sep 03, 2021
A clipboard where a user can add and retrieve multiple items to and from (resp) from the clipboard cache.

A clipboard where a user can add and retrieve multiple items to and from (resp) from the clipboard cache.

Gaurav Bhattacharjee 2 Feb 07, 2022
Provide Prometheus url_sd compatible API Endpoint with data from Netbox

netbox-plugin-prometheus-sd Provide Prometheus http_sd compatible API Endpoint with data from Netbox. HTTP SD is a new feature in Prometheus and not a

Felix Peters 66 Dec 19, 2022
pyForgeCert is a Python equivalent of the original ForgeCert written in C#.

pyForgeCert is a Python equivalent of the original ForgeCert written in C#.

Evi1cg 47 Oct 08, 2022
This repo houses the qhub frontend moving forward.

This repo houses the qhub frontend moving forward. This effort will house a backend written in fastAPI, and a fronend in Vue, with additional components.

Quansight 1 Feb 10, 2021
Hospitality app for ERPNext to manage hotels & restaurants.

Hospitality ERPNext Hospitality module is designed to handle workflows for Hotels and Restaurants. Manage Restaurants The Restaurant module in ERPNext

Frappe 19 Dec 26, 2022
Comprehensive OpenAPI schema generator for Django based on pydantic

🗡️ Djagger Automated OpenAPI documentation generator for Django. Djagger helps you generate a complete and comprehensive API documentation of your Dj

13 Nov 26, 2022
Basit bir cc generator'ü.

Basit bir cc generator'ü. Setup What To Do; Python Installation We install python from CLICK Generator Board After installing the file and python, we

Lâving 7 Jan 09, 2022
An Embedded Linux Project Build and Compile Tool -- An Bitbake UI Extension

Dianshao - An Embedded Linux Project Build and Compile Tool

0 Mar 27, 2022
A one place destination to check whatever is trending on the top social and news websites at present.

UpTrend A one place destination to check whatever is trending on the top social and news websites at present. Explore the docs » View Demo · Report Bu

Google Developer Student Clubs - JGEC 10 Oct 03, 2021