479 Commits

Author SHA1 Message Date
Fringg
bfef7cc629 fix: prevent race condition expiring active daily subscriptions
MonitoringService._check_expired_subscriptions() was marking daily
subscriptions as expired before DailySubscriptionService could charge
and extend them. Now get_expired_subscriptions() excludes active
(non-paused) daily subs — they are managed by DailySubscriptionService.

Also fix cabinet "0m until next charge" display: return None when
next_daily_charge_at is in the past instead of a stale datetime.
2026-02-25 15:07:24 +03:00
Fringg
f300e07ce2 chore: ruff format 2026-02-25 06:34:22 +03:00
Fringg
c1da8a4dba fix: RBAC audit log action filter and legacy admin level
- Change audit log action filter from exact match to ILIKE substring
  search so admins can search by partial action names
- Return level 1000 (not 999) for legacy config-based admins in
  /me/permissions so frontend correctly enables role management buttons
2026-02-25 04:07:09 +03:00
Fringg
4598c2785a fix: RBAC API response format fixes and audit log user info
- Simplify permission registry to return flat list[PermissionSection] with actions as list[str]
- Add user_first_name and user_email to audit log entries via selectinload
- Fix unused import and naming convention lint warnings
2026-02-25 03:40:40 +03:00
Fringg
bc7d0612f1 fix: specify foreign_keys on User.admin_roles_rel to resolve ambiguous join
UserRole has two FKs to users (user_id and assigned_by), causing
SQLAlchemy AmbiguousForeignKeysError on mapper initialization.
2026-02-25 03:23:41 +03:00
Fringg
3fee54f657 feat: add RBAC + ABAC permission system for admin cabinet
Backend:
- 4 new models: AdminRole, UserRole, AccessPolicy, AdminAuditLog
- Permission engine with RBAC wildcard matching + ABAC policy evaluation
- 26 permission sections (78 unique permissions) covering all admin routes
- require_permission() FastAPI dependency for route-level access control
- JWT tokens carry permissions, roles, role_level for frontend checks
- Admin roles CRUD with level-based hierarchy (viewers → superadmin)
- ABAC policies with time ranges and IP whitelist conditions
- Full audit log with CSV export
- Bootstrap service seeds 5 preset roles and assigns superadmins at startup
- Alembic migration 0011 for all RBAC tables
2026-02-25 03:02:40 +03:00
Fringg
a594a0f79f fix: improve campaign notifications and ticket media in admin topics
- Campaign notifications: add tariff bonus display, hide empty promo group,
  compact format matching purchase notification style
- Ticket notifications: send media (photos) in the same topic as the text
  notification instead of separately. Uses caption for short texts, sequential
  messages for long texts with correct message_thread_id routing
2026-02-25 00:44:44 +03:00
Fringg
3642462670 feat: add per-channel disable settings and fix CHANNEL_REQUIRED_FOR_ALL bug
- Fix critical bug: is_active_paid_subscription() guard was blocking
  CHANNEL_REQUIRED_FOR_ALL from disabling paid subscriptions
- Add disable_trial_on_leave and disable_paid_on_leave columns to
  RequiredChannel model with Alembic migration 0010
- Refactor enforcement logic in channel_member.py and channel_checker.py
  to use per-channel settings instead of global env vars
- Update CRUD, Pydantic schemas, and admin API routes for new fields
- Add should_disable_subscription() and get_channel_settings() to
  channel_subscription_service for per-channel decision logic
2026-02-25 00:24:31 +03:00
Fringg
a7db469fd7 fix: remove @username channel ID input, auto-prefix -100 for bare digits
@username resolution via bot.get_chat() was unreliable for subscription
checking. Now only numeric channel IDs are accepted with automatic -100
prefix when entering bare digits (e.g. 1234567890 -> -1001234567890).
2026-02-24 03:06:57 +03:00
Fringg
8375d7ecc5 feat: add multi-channel mandatory subscription system
- Multi-channel subscription enforcement via middleware, events, and cabinet API
- 3-layer cache architecture: Redis -> PostgreSQL -> rate-limited Telegram API
- ChatMemberUpdated event-driven tracking with automatic VPN access control
- Admin management via bot FSM handler and REST API with full CRUD
- Channel ID normalization: @username resolved to numeric ID at creation time
- Fail-closed error handling: API errors deny access (security-first)
- Background reconciliation with keyset pagination (100 per batch)
- Per-user rate limiting on subscription check button (5s cooldown)
- Redis connection pooling via cache singleton (no per-request connections)
- Database: channel_id index, multi-row upsert optimization
- Localization: en, ru, zh, fa, ua translations for all new strings
- Frontend blocking UI with channel list and subscription status
- Admin channel management page with toggle, delete, and create
2026-02-24 02:50:31 +03:00
Fringg
1b6bbc7131 fix: protect active paid subscriptions from being disabled in RemnaWave
Add is_active_paid_subscription() helper that checks if subscription is
non-trial, active, and not expired. Use it across all disable_remnawave_user
call sites to prevent disabling VPN access for users with paid subscriptions.

Protected paths: block_user, delete_user_account, broadcast cleanup,
channel unsubscribe, admin deactivation, webapi endpoints, cabinet
reset-trial, reset-subscription, and disable-user endpoints.
2026-02-23 16:49:31 +03:00
Fringg
18c2477173 feat: add referral code tracking to all cabinet auth methods + email_templates migration
Referral links from cabinet (?ref=CODE) were only tracked for email registration.
Now referral_code is accepted and processed in Telegram initData, Telegram Widget,
and OAuth authentication endpoints. Includes self-referral protection by email
for OAuth, proper error logging, and the missing email_templates table migration.
2026-02-18 23:59:29 +03:00
Fringg
a7f3d652c5 fix: use AwareDateTime TypeDecorator for all datetime columns
TypeDecorator with process_result_value guarantees naive datetimes
from pre-TIMESTAMPTZ databases are converted to UTC-aware on every
load. Replaces unreliable event listener approach. All 175 DateTime
columns now use AwareDateTime.
2026-02-18 11:11:58 +03:00
Fringg
f7d33a7d2b fix: auto-convert naive datetimes to UTC-aware on model load
SQLAlchemy event listener on Base ensures all DateTime columns are
timezone-aware after loading from DB. Fixes TypeError crashes in
50+ comparison sites across handlers, services, and middlewares
for pre-TIMESTAMPTZ databases.
2026-02-18 11:01:04 +03:00
Fringg
bd11801467 fix: extend naive datetime guard to all model properties
Move _aware() to module level and apply to 4 more models:
- PromoCode.is_valid (valid_from, valid_until)
- TrafficPurchase.is_expired (expires_at)
- CabinetRefreshToken.is_expired (expires_at)
- Ticket.is_user_reply_blocked (user_reply_block_until)
2026-02-18 10:44:13 +03:00
Fringg
e512e5fe6e fix: handle naive datetimes in Subscription properties
Databases that haven't run the TIMESTAMPTZ migration return naive
datetimes from end_date. Comparing with datetime.now(UTC) raises
TypeError. Added _aware() helper to normalize naive→aware in
is_active, is_expired, should_be_expired, actual_status, days_left,
time_left_display, and extend_subscription.
2026-02-18 10:36:46 +03:00
Fringg
68499ee043 chore: ruff format 2026-02-18 09:51:56 +03:00
Fringg
0c07812ecc feat: add campaign_id to ReferralEarning for campaign attribution
Adds nullable FK campaign_id to referral_earnings table, enabling
direct campaign ROI analytics without JOINing through registrations.

- Model: campaign_id column + AdvertisingCampaign relationship
- CRUD: get_user_campaign_id() helper, campaign_id param in create_referral_earning
- Service: resolve campaign_id in all earning creation paths
- Cabinet API: campaign_name in earnings response
- Migration 0002: add column + deterministic backfill via DISTINCT ON
2026-02-18 09:12:01 +03:00
Fringg
764e063bfe style: apply ruff formatting 2026-02-18 08:11:33 +03:00
Fringg
784616b349 refactor: replace universal_migration.py with Alembic
Remove the 7,791-line universal_migration.py and 16 incomplete individual
Alembic migrations. Replace with a single initial schema migration using
Base.metadata.create_all(checkfirst=True).

Changes:
- Add programmatic Alembic runner (app/database/migrations.py) with
  auto-stamp logic for existing databases transitioning from
  universal_migration
- Extract ensure_default_web_api_token() to web_api_token_service.py
- Extract sync_postgres_sequences() to database.py with SQL injection
  prevention via _quote_ident()
- Add HMAC token hashing support with backward-compatible dual-hash
  fallback and automatic rehashing
- Remove dead init_db() function and unused imports
- Add Makefile targets: migrate, migration, migrate-stamp, migrate-history
- Fix fileConfig() destroying structlog config (disable_existing_loggers)
- Remove duplicate migrations/alembic/alembic.ini with credentials
- Add script.py.mako template for future migration generation
- Update startup flow: alembic upgrade → sync sequences → ensure token
- Harden database.py: ParamSpec for retry decorator, safe URL logging,
  echo='debug' mode, execute_with_retry validation
- Update documentation references

31 files changed, 302 insertions(+), 9,226 deletions(-)
2026-02-18 08:10:20 +03:00
Fringg
b4b10c998c fix: add blocked_count column migration to universal_migration.py
The column existed in the SQLAlchemy model and Alembic migration but was
missing from universal_migration.py which is used for auto-migrations on
startup, causing "column broadcast_history.blocked_count does not exist"
error in the broadcasts admin page.
2026-02-18 06:57:03 +03:00
Fringg
366df18c54 feat: enforce 1-to-1 partner-campaign binding with partner info in campaigns
- Add partner_user_id/partner_name to campaign list and detail responses
- Add partner_user_id to campaign create/update schemas
- Add GET /available-partners endpoint for partner dropdown
- Atomic assign with UPDATE...WHERE to prevent race conditions
- Validate partner exists and is approved in create/update
- Set updated_at on assign/unassign operations
- Eager-load partner relationship in campaign queries
2026-02-18 06:47:02 +03:00
Fringg
10e231e52e feat: blocked user detection during broadcasts, filter blocked from all notifications
- Broadcast tri-state return: 'sent'/'blocked'/'failed' with blocked_count tracking
- Background cleanup: mark blocked users + disable their subscriptions + Remnawave
- blocked_count in BroadcastHistory model, schemas, API responses, admin UI
- Filter User.status==ACTIVE in subscription queries: get_expiring_subscriptions,
  get_expired_subscriptions, get_subscriptions_for_autopay,
  get_daily_subscriptions_for_charge, get_disabled_daily_subscriptions_for_resume
- Guard in notification_delivery_service.send_notification for BLOCKED/DELETED users
- Fix subscription tariff switch: preserve remaining days with total_seconds()
- Fix redundant local UTC imports across 16 files
- Fix test mocks: add **kwargs, correct assertion, remove dead expression
2026-02-17 18:37:25 +03:00
Fringg
88997492c3 fix: critical security and data integrity fixes for partner system
- Add SELECT FOR UPDATE locking on all financial state transitions
  (withdrawal approve/reject/complete/create, partner approve/reject)
- Add html.escape() on all user-controlled values in email templates
- Wrap sync SMTP send_email in asyncio.to_thread to avoid blocking event loop
- Add missing database indexes on referral_earnings(user_id, referral_id),
  users(referred_by_id, partner_status), withdrawal_requests(user_id, status),
  advertising_campaigns(partner_user_id)
2026-02-17 12:28:30 +03:00
Fringg
acc1323a54 fix: move PartnerStatus enum before User class to fix NameError
PartnerStatus was defined after the User class that references it,
causing a NameError on startup.
2026-02-17 09:56:11 +03:00
Fringg
58bfaeaddb feat: add partner system and withdrawal management to cabinet
- Partner application flow: user applies, admin reviews/approves/rejects
- Individual commission % per partner with admin management
- Campaign assignment/unassignment to partners
- Withdrawal system: balance check, create request, cancel
- Admin withdrawal management with risk scoring and fraud analysis
- Database migration: partner_applications table, user partner fields, campaign partner_user_id
- Pydantic schemas with proper validation bounds
- Batch user fetching to prevent N+1 queries
- Row locking on cancel to prevent race conditions
2026-02-17 09:51:36 +03:00
Fringg
df5415f30b fix: reorder button_click_logs migration to nullify before ALTER TYPE
ALTER COLUMN user_id TYPE INTEGER failed with "integer out of range"
because the column contained telegram_id values (BIGINT) exceeding
INTEGER max. Swapped order: SET NULL first, then ALTER TYPE.
2026-02-17 08:19:21 +03:00
Fringg
d9552799c1 feat: add web campaign links with bonus processing in auth flow
- Add web_link generation for campaigns (uses MINIAPP_CUSTOM_URL)
- Process campaign_slug in all auth endpoints (telegram, widget, email, oauth)
- Apply campaign bonus (balance/subscription/tariff) with SELECT FOR UPDATE lock
- Add rollback + user refresh on campaign bonus failure
- Fix N+1 query in campaign registrations (batch subscription check)
- Remove duplicate queries in get_campaign_statistics (~60 lines dead code)
- Simplify _store_refresh_token (remove TOCTOU pre-check, keep IntegrityError)
- Remove dead expression in campaign_service.py
- Align start_parameter max_length to 64 (matches DB column)
- Remove unused campaign_slug from EmailRegisterStandaloneRequest
2026-02-17 06:44:03 +03:00
Fringg
eb18994b7d fix: complete datetime.utcnow() → datetime.now(UTC) migration
- Migrate 660+ datetime.utcnow() across 153 files to datetime.now(UTC)
- Migrate 30+ datetime.now() without UTC to datetime.now(UTC)
- Convert all 170 DateTime columns to DateTime(timezone=True)
- Add migrate_datetime_to_timestamptz() in universal_migration with SET LOCAL timezone='UTC' safety
- Remove 70+ .replace(tzinfo=None) workarounds
- Fix utcfromtimestamp → fromtimestamp(..., tz=UTC)
- Fix fromtimestamp() without tz= (system_logs, backup_service, referral_diagnostics)
- Fix fromisoformat/isoparse to ensure aware output (platega, yookassa, wata, miniapp, nalogo)
- Fix strptime() to add .replace(tzinfo=UTC) (backup_service, referral_diagnostics)
- Fix datetime.combine() to include tzinfo=UTC (remnawave_sync, traffic_monitoring)
- Fix datetime.max/datetime.min sentinels with .replace(tzinfo=UTC)
- Rename panel_datetime_to_naive_utc → panel_datetime_to_utc
- Remove DTZ003 from ruff ignore list
2026-02-17 04:45:40 +03:00
Fringg
ff21b27b98 fix: address remaining abs() issues from review
- admin_traffic._get_bulk_spending: add func.abs() for SUBSCRIPTION_PAYMENT SUM
- get_user_total_spent_kopeks: move abs() from Python to SQL (per-row func.abs)
- referral_contest.total_outside: add abs() for mixed-type sum
- Revert func.abs() from generic by_type aggregation to preserve refund/withdrawal signs
2026-02-17 03:47:39 +03:00
Fringg
4247981c98 fix: normalize transaction amount signs across all aggregations
SUBSCRIPTION_PAYMENT transactions have inconsistent signs in DB
(some negative, some positive). Add func.abs()/abs() to all SUM
queries and display code to ensure correct totals regardless of sign.

Affected: admin statistics, referral contest stats, tariff revenue,
campaign stats, reporting service, admin renewal notifications.
2026-02-17 03:40:37 +03:00
Fringg
c30972f6a7 fix: prevent negative amounts in spent display and balance history
SUBSCRIPTION_PAYMENT transactions are stored with negative amount_kopeks.
- get_user_total_spent_kopeks now returns abs() to fix "Потрачено: -155 ₽"
  and broken promo group threshold comparisons
- Balance history uses abs() before format_price to prevent "--85 ₽"
2026-02-17 03:36:56 +03:00
Fringg
a93a32f3a7 fix: resolve MissingGreenlet error when accessing subscription.tariff
Add .selectinload(Subscription.tariff) chain to all User queries that
load subscriptions, preventing lazy loading of the tariff relationship
in async context. Also replace unsafe getattr(subscription, 'tariff')
with explicit async get_tariff_by_id() in handle_extend_subscription.
2026-02-16 17:54:43 +03:00
Fringg
8a6650e57c fix: suppress startup log noise (~350 lines → ~30)
- Suppress migration logger to WARNING during startup (main.py)
- Remove debug logs from get_traffic_packages() leaking before structlog init
- Downgrade handler registration logs to debug (start.py)
- Remove duplicate section headers from migration orchestrator
2026-02-16 09:34:17 +03:00
Fringg
1f0fef114b refactor: complete structlog migration with contextvars, kwargs, and logging hardening
- Add ContextVarsMiddleware for automatic user_id/chat_id/username binding
  via structlog contextvars (aiogram) and http_method/http_path (FastAPI)
- Use bound_contextvars() context manager instead of clear_contextvars()
  to safely restore previous state instead of wiping all context
- Register ContextVarsMiddleware as outermost middleware (before GlobalError)
  so all error logs include user context
- Replace structlog.get_logger() with structlog.get_logger(__name__) across
  270 calls in 265 files for meaningful logger names
- Switch wrapper_class from BoundLogger to make_filtering_bound_logger()
  for pre-processor level filtering (performance optimization)
- Migrate 1411 %-style positional arg logger calls to structlog kwargs
  style across 161 files via AST script
- Migrate log_rotation_service.py from stdlib logging to structlog
- Add payment module prefixes to TelegramNotifierProcessor.IGNORED_LOGGER_PREFIXES
  and ExcludePaymentFilter.PAYMENT_MODULES to prevent payment data leaking
  to Telegram notifications and general log files
- Fix LoggingMiddleware: add from_user null-safety for channel posts,
  switch time.time() to time.monotonic() for duration measurement
- Remove duplicate logger assignments in purchase.py, config.py,
  inline.py, and admin/payments.py
2026-02-16 09:18:12 +03:00
Fringg
97ec39aa80 fix: add promo code anti-abuse protections
- Rate-limit on brute-force: 5 failed attempts per 5 min blocks user
- Daily stacking limit: max 5 promo activations per 24h (in-memory + DB)
- Format validation: only alphanumeric/hyphen/underscore, 3-50 chars
2026-02-16 06:52:45 +03:00
Fringg
80914c1af7 fix: daily tariff subscriptions stuck in expired/disabled with no resume path
- Keyboard now shows "Возобновить" for disabled/expired daily tariffs
  instead of useless "Приостановить"
- resume_daily_subscription handles EXPIRED→ACTIVE (not only DISABLED)
- Pause handler detects inactive status and calls resume directly
- subscription_extend redirects daily tariffs to subscription info
  (daily tariffs have no period_prices, so extend page was empty)
2026-02-15 23:17:45 +03:00
Fringg
4048aebb9f chore: format models.py 2026-02-12 21:08:05 +03:00
Fringg
bfd66c42c1 fix: add passive_deletes to Subscription relationships to prevent NOT NULL violation on cascade delete 2026-02-12 20:59:28 +03:00
Fringg
57dc1ff47f fix: resolve deadlock on server_squads counter updates and add webhook notification toggles
- Fix deadlock: enforce sorted lock ordering in add_user_to_servers/remove_user_from_servers
- Fix cross-call deadlock: add update_server_user_counts() for atomic add+remove in one sorted pass
- Fix deadlock in squad migration: use sorted dict iteration for counter updates
- Fix broken "Buy traffic" button: subscription_add_traffic → buy_traffic callback_data
- Add 12 webhook notification toggle settings (WEBHOOK_NOTIFY_*) with master toggle
- Add admin UI category "Уведомления от вебхуков" with hints in BotConfigurationService
- Add toggle check in _notify_user() respecting master and per-event settings
2026-02-12 06:47:26 +03:00
Fringg
c5124b97b6 fix: payment race conditions, balance atomicity, renewal rollback safety
- YooKassa: SELECT FOR UPDATE on payment row to prevent concurrent double-processing
- subtract_user_balance: row locking to prevent concurrent balance race conditions
- subtract_user_balance: transaction creation before commit for atomicity
- subscription renewal: compensating refund if extend_subscription fails after charge
- StaleDataError: use savepoint instead of full rollback to protect parent transaction
2026-02-11 21:49:37 +03:00
Fringg
fcaa9dfb27 fix: clean stale squad UUIDs from tariffs during server sync
When squads are deleted from the RemnaWave panel and servers are synced,
the bot cleaned subscription connected_squads but left stale UUIDs in
tariff.allowed_squads. This caused errors when users tried to purchase
or extend subscriptions with tariffs referencing deleted squads.

Now sync_with_remnawave also removes stale UUIDs from all tariffs.
2026-02-11 18:37:19 +03:00
Fringg
c30c2feee1 fix: handle StaleDataError in webhook user.deleted server counter decrement
When a user is deleted from the panel, the subscription may already be
cascade-deleted by the time the webhook handler tries to decrement
server counters. This caused StaleDataError followed by
PendingRollbackError when accessing subscription.id in the error handler.

- Save subscription.id before DB operations to avoid lazy load after rollback
- Catch StaleDataError explicitly and rollback the session
- Re-fetch subscription/user after potential rollback in _handle_user_deleted
- Skip subscription cleanup if it was already cascade-deleted
2026-02-11 18:35:36 +03:00
Fringg
887ea9cf5a style: format subscription.py with ruff 2026-02-11 04:45:42 +03:00
Fringg
bee4aa4284 fix: protect server counter callers and fix tariff change detection
- Wrap unprotected add/remove_user_to/from_servers calls in try/except
  in miniapp.py and cabinet subscription.py to prevent 500 errors
- Fix is_tariff_change to include classic-to-tariff transitions
  (subscription.tariff_id=None → new tariff_id) so purchased traffic
  is properly reset when switching modes
2026-02-11 04:44:15 +03:00
Fringg
b167ed3dd1 fix: preserve purchased traffic when extending same tariff
extend_subscription was unconditionally resetting purchased_traffic_gb
and deleting TrafficPurchase records whenever traffic_limit_gb was passed,
even when extending the same tariff (not changing). Now only resets
on actual tariff change (is_tariff_change=True), preserving purchased
traffic on same-tariff extensions.
2026-02-11 04:38:08 +03:00
Fringg
6cec024e46 fix: use flush instead of commit in server counter functions
add_user_to_servers and remove_user_from_servers were calling
db.commit() internally, breaking transaction atomicity for all
callers that perform additional operations afterward. Changed to
db.flush() so the caller controls the commit boundary.
2026-02-11 04:15:50 +03:00
Fringg
e94b93d0c1 fix: handle nullable traffic_limit_gb and end_date in subscription model
Add None-safety guards to Subscription model properties (is_active,
is_expired, should_be_expired, actual_status, days_left,
traffic_used_percent) and pricing handler comparisons to prevent
TypeError when nullable columns contain None values.
2026-02-10 20:35:42 +03:00
Fringg
184c52d4ea feat: webhook protection — prevent sync/monitoring from overwriting webhook data
Add last_webhook_update_at timestamp to Subscription model. When a webhook
handler modifies a subscription, it stamps this field. Auto-sync, monitoring,
and force-check services skip subscriptions updated by webhook within the
last 60 seconds, preventing stale panel data from overwriting fresh
real-time changes.

- Add last_webhook_update_at column + migration
- Stamp all 8 webhook handlers with commit in every code path
- Add is_recently_updated_by_webhook() guard in 12 sync/monitoring paths
- Add REMNAWAVE_WEBHOOK_* variables to .env.example
- Add webhook setup documentation to README with Caddy/nginx examples
- Fix pre-existing yookassa webhook test (mock AsyncSessionLocal)
2026-02-10 07:16:22 +03:00
Fringg
90d9df8f0e fix: preserve payment initiation time in transaction created_at
Transaction created_at and completed_at showed identical timestamps
because webhook handlers created transactions with is_completed=True
in a single step. Now all 10 payment providers pass payment.created_at
to the transaction so created_at reflects when the user initiated
the payment, not when the webhook processed it.

Also: remove duplicate datetime import in inline.py, upgrade button
stats DB error logging from debug to warning, add index on
button_click_logs.button_type for analytics queries.
2026-02-10 04:26:23 +03:00