Skip to main content

Posts

Database-Logger-PyODBC

Database-Logger-PyODBC Overview This repository contains a Python implementation of a logging system that supports: Database Logging : Inserts log entries into a SQL Server database. File-Based Logging : Maintains error logs in a local file as a backup mechanism. Automatic Retry for Failed Logs : Attempts to process and insert failed log entries from the local log file into the database. Features Database Integration : Logs critical events directly into a SQL Server database using  pyodbc . Provides detailed logging fields such as timestamp, task ID, log type, status, and error details. File Backup : Writes logs to a local file ( dbErrorLog.txt ) in case of database connection issues. Retries inserting failed log entries from the file to the database. Error Handling : Catches and logs errors encountered during database operations. Ensures no logs are lost by falling back to file-based storage. Prerequisites Python 3.8+. Required Python packages: pyodbc logging SQL Server database w...
Post a Comment
Read more
Recent posts

Data-Automation-SFTP-Database

Data-Automation-SFTP-Database Overview This repository contains a Python script for automating data workflows involving: SFTP File Download:  Securely connecting to an SFTP server to download files. Data Preprocessing:  Cleaning and transforming data from CSV files, including handling numeric, currency, and datetime columns. Database Integration:  Uploading cleaned data to an SQL database. Logging and Notifications:  Comprehensive logging and email alerts for status updates and error reporting. Features SFTP Automation : Securely connects to an SFTP server. Downloads files based on specific criteria. Archives downloaded files in a remote archive directory. Data Cleaning : Handles invalid and missing values ( NULL ,  N/A , etc.). Processes numeric and currency columns for database compatibility. Parses and standardizes datetime fields. Database Upload : Uses SQLAlchemy for seamless database integration. Supports MSSQL with ODBC Driver 17. Error Handling & Not...
Post a Comment
Read more

MySQL Table Data Export And Send To Email

MySQL Data Downloader and Emailer This project connects to a MySQL database, downloads data from specified tables, stores it in CSV files, and sends them via email. Prerequisites Python 3.x MySQL Connector for Python Pandas SMTP access for sending emails Setup Clone the repository: Navigate to the project directory: Install the required Python packages: Code:  import mysql.connector import pandas as pd import os from datetime import datetime import smtplib from email.mime.multipart import MIMEMultipart from email.mime.base import MIMEBase from email.mime.text import MIMEText from email import encoders # Database connection details db_config = {     'host': 'xx.xx.xx.xx',  # Replace with your MySQL host     'user': 'xxxxxxxx',  # Replace with your MySQL username     'password': 'xxxxxxxx',  # Replace with your MySQL password     'database': 'xxxxxxxx'  # Replace with your database name } # List of table names to query table...
Post a Comment
Read more