DebasisNandiBlog

Database-Logger-PyODBC Overview This repository contains a Python implementation of a logging system that supports: Database Logging : Inserts log entries into a SQL Server database. File-Based Logging : Maintains error logs in a local file as a backup mechanism. Automatic Retry for Failed Logs : Attempts to process and insert failed log entries from the local log file into the database. Features Database Integration : Logs critical events directly into a SQL Server database using  pyodbc . Provides detailed logging fields such as timestamp, task ID, log type, status, and error details. File Backup : Writes logs to a local file ( dbErrorLog.txt ) in case of database connection issues. Retries inserting failed log entries from the file to the database. Error Handling : Catches and logs errors encountered during database operations. Ensures no logs are lost by falling back to file-based storage. Prerequisites Python 3.8+. Required Python packages: pyodbc logging SQL Server database w...

Data-Automation-SFTP-Database Overview This repository contains a Python script for automating data workflows involving: SFTP File Download:  Securely connecting to an SFTP server to download files. Data Preprocessing:  Cleaning and transforming data from CSV files, including handling numeric, currency, and datetime columns. Database Integration:  Uploading cleaned data to an SQL database. Logging and Notifications:  Comprehensive logging and email alerts for status updates and error reporting. Features SFTP Automation : Securely connects to an SFTP server. Downloads files based on specific criteria. Archives downloaded files in a remote archive directory. Data Cleaning : Handles invalid and missing values ( NULL ,  N/A , etc.). Processes numeric and currency columns for database compatibility. Parses and standardizes datetime fields. Database Upload : Uses SQLAlchemy for seamless database integration. Supports MSSQL with ODBC Driver 17. Error Handling & Not...