Working in enterprise software development, especially on backbone systems like ERP, inevitably molds one into a certain way of thinking. For years, I've coded and managed countless flows in a manufacturing company's ERP, from purchasing to production, shipping to invoicing. During these processes, I've repeatedly seen how dozens of modules communicate with each other and how a small change can create a domino effect in another module. Deciphering the anatomy of these complex integrations has become a habit not only in my job but also, unconsciously, in my own side projects.
While this habit sometimes leads me to overcomplicate things, it often provides valuable insight into how systems work deeply. I want to share the situations that arise when the heavy and detailed thinking of the corporate world seeps into my small projects, and the lessons I've learned from them. This is not just a technical topic, but also an observation of the impact of professional experiences on personal creativity.
Modularity Myths and Corporate Realities
In an ERP system, the word "modularity" often sounds appealing, but in reality, the boundaries of this concept are quite blurred. Modules like Finance, Inventory, Production, and Sales may seem independent, but they are actually in constant and deep data exchange. An inventory movement instantly creating an accounting voucher, a production order consuming material stock, or a sales order triggering shipment planning are just a few examples of these invisible ties. These ties are critical for system consistency.
One of the most common situations I encountered in my production ERP was a performance bottleneck in one module completely locking up another critical workflow. For example, during one period, inventory counting operations in the Inventory Management module delayed the creation of vouchers in the Accounting module by up to 30 minutes. This situation led to critical financial reports not being available on time, especially during month-end closings. The root of the problem was that inventory movements were directly triggered into the accounting voucher table with a single INSERT operation, and this table had grown to enormous sizes and was not properly indexed.
-- An operation expected to be fast could slow down on large tables.
INSERT INTO muhasebe_fisleri (
fis_tarihi, belge_no, aciklama, borc_tutari, alacak_tutari, hesap_kodu, referans_id, referans_tipi
)
SELECT
s.hareket_tarihi, s.belge_no, 'Stok Hareketi ' || s.hareket_kodu, s.tutar, 0,
CASE WHEN s.hareket_tipi = 'GIRIS' THEN '153' ELSE '600' END,
s.id, 'STOK_HAREKET'
FROM
stok_hareketleri s
WHERE
s.muhasebelesme_durumu = 'BEKLIYOR' AND s.hareket_tarihi < CURRENT_DATE
-- And this query scanned hundreds of thousands of rows every time it ran.
;
Such problems demonstrate how intertwined modules are, not just at the API level, but also at the database schema and business rule levels. The solution didn't come with a simple ALTER INDEX or VACUUM ANALYZE; I had to make the accounting process for inventory movements asynchronous using the transaction outbox pattern. This significantly improved the overall system performance and responsiveness while reducing module dependencies. While modularity isn't an illusion, understanding the challenges brought by deep integration was one of the biggest lessons in this area.
ℹ️ Asynchronous Processing Philosophy
In enterprise systems, especially for critical operations involving different domains, asynchronous processing models (e.g., the
transaction outboxpattern) are indispensable not only for performance improvement but also for system flexibility and fault tolerance. The failure of one module does not directly affect others, allowing us to build more robust systems.
Corporate Logic Seeping into My Side Projects
This "deep integration" perspective I gained in the corporate ERP world inevitably seeps into my own side projects. Even when developing a simple task management application, I tend to build a complex hierarchy where each task is associated with a tag, this tag belongs to a project, and the project is linked to a customer or category. It's as if this is inevitable for future "reporting" or "analysis" needs.
For example, when writing the backend for my own task management application, even for a simple task creation process, I designed an event table for future event sourcing. task_created event, task_updated event... as if it would have tens of thousands of users. Even when I was the only user, I had entered a loop of firing an event to change a task's status, then processing this event with an event handler to update the main table.
# Task creation service - an unnecessarily complex structure initially
from datetime import datetime
class TaskCreatedEvent:
def __init__(self, task_id: str, title: str, user_id: str, created_at: datetime):
self.task_id = task_id
self.title = title
self.user_id = user_id
self.created_at = created_at
class TaskService:
def create_task(self, title: str, user_id: str):
task_id = f"task_{datetime.now().timestamp()}"
event = TaskCreatedEvent(task_id, title, user_id, datetime.now())
self._publish_event(event) # Save the event to a queue or database
print(f"Task created event published for {task_id}")
return task_id
def _publish_event(self, event):
# In reality, this part could be a database write or a Kafka queue.
# In my case, I was saving it as JSON to a simple SQLite table.
pass
# Main application flow
task_service = TaskService()
new_task_id = task_service.create_task("Finish blog post", "mustafa.e")
# Then a separate thread/process would pick up this event and write the main task to the database.
This approach was intended to bring the benefits of patterns like event-driven architecture and CQRS (scalability, flexibility, auditing) from corporate software to my side projects. However, in a solo project, the cognitive load and development overhead introduced by these extra layers often outweighed the benefits. Eventually, when I reverted to a simple CRUD architecture, I progressed faster, and the project became much easier to maintain. This showed me that monolith vs microservice choices apply not only to large projects but even to my own small ones.
The Invisible Costs of Integration: More Than Just Technical Debt
The integration challenges we face in corporate projects are often discussed under the concept of "technical debt." However, for me, beyond that, there are also personal time and mental energy costs. I have a financial calculator side project running on my own VPS. Initially, it only needed to perform simple currency conversions. But with a corporate mindset, I didn't leave it as just "currency conversion." I added modules such as fetching data from different sources for exchange rates (CBRT, free market, crypto exchanges), archiving historical exchange rate data, and calculating average exchange rates for different date ranges.
Each of these additional modules required its own data source integration, API rate limiting management, and data consistency control. As a result, a simple currency conversion API turned into a "mini-ERP" that communicated with 5 different services in the background and used 3 different databases (PostgreSQL, Redis cache, InfluxDB experiments for time series). Naturally, this complexity brought N+1 query problems with it. Last month, a simple historical exchange rate query took more than 2 seconds on my PostgreSQL server. When I looked at the journald logs, I saw that PostgreSQL was overflowing with idle in transaction warnings.
Jun 09 03:14:21 my-vps postgres[12345]: LOG: duration: 2154.321 ms execute <unnamed>: SELECT * FROM exchange_rates WHERE currency_pair = $1 AND date BETWEEN $2 AND $3 ORDER BY date ASC;
Jun 09 03:14:21 my-vps postgres[12345]: DETAIL: Parameters: $1 = 'USDTRY', $2 = '2025-01-01', $3 = '2025-12-31'
Jun 09 03:14:21 my-vps postgres[12345]: LOG: statement: SELECT * FROM exchange_rates WHERE currency_pair = 'USDTRY' AND date BETWEEN '2025-01-01' AND '2025-12-31' ORDER BY date ASC;
This log entry indicated that I needed to thoroughly optimize my PostgreSQL performance. I needed to perform connection pool tuning, review my index strategies (perhaps try BRIN index), and pay more attention to vacuum monitoring. Such issues can lead to WAL bloat, replication lag, or OOM eviction in a production environment. Having to manage this complexity even in my own side project must be a "professional deformation" brought on by the corporate world. I had experienced a similar trade-off during a VPS migration process; trying to transform a simple monolithic structure into a distributed one, operational complexities took much more of my time than expected.
Zero-Trust Architecture and Integration Boundaries
Zero-Trust Architecture (ZTNA) principles are a fundamental approach to security in large enterprise networks and microservice architectures. The "never trust, always verify" logic requires every module or service to be authorized and authenticated against each other, even if they are on the same network. This approach is supported by steps such as VLAN segmentation, routing authentication, and egress control in network security.
I try to apply these principles even to my own side projects. For example, in my Android spam application, I tried to ensure that different internal components (message analysis engine, number database query tool, reporting service) use specific API keys or JWT tokens when communicating with each other. Yes, these are components running within the same application, but my corporate experience has always taught me to consider "threats that can come from within."
# A simple internal API call, but with an attempt at token authorization
import requests
def get_spam_score(phone_number: str, token: str):
headers = {"Authorization": f"Bearer {token}"}
response = requests.get(
"http://localhost:8000/api/v1/spam_analysis",
params={"phone": phone_number},
headers=headers
)
response.raise_for_status()
return response.json()["score"]
# Token generation within the main application (simple example)
internal_api_token = "my_super_secret_internal_token" # In reality, it would be a JWT/OAuth2 flow
score = get_spam_score("5551234567", internal_api_token)
print(f"Spam Score: {score}")
Such approaches stem from my experience securing inter-microservice communication using JWT/OAuth2 patterns in a client project. There, each service verified the authorization of the request it received from the token in the Authorization header. This, along with rate limiting and DDoS mitigation layers, made the system more resilient to external attacks. Although there's no need for this level of security in my small projects, applying these principles feels like both good practice and preparation for potential future growth. However, of course, these additional security layers can also introduce latency and overhead, which is always a trade-off.
The Power of Simplicity: Lessons Learned and Future Approach
After years of corporate integration and architectural experience, the biggest lesson I've learned in my personal side projects is that simplicity is often more valuable than complexity. Starting every project with event sourcing, CQRS, or microservice architectures as if it will have hundreds of thousands of users tomorrow often creates unnecessary overhead. Especially in projects where I work alone, this reduces my development speed and increases my cognitive load.
Now, when starting a new side project, I first consider the simplest CRUD (Create, Read, Update, Delete) model. I only turn to more complex solutions when I genuinely need them, for example, when a deeper integration is required via a Flutter native bridge in a specific part of my mobile application, or when I encounter a specific metadata reject problem during Play Store publishing processes. I also keep observability (metrics, logs, traces) at a very basic level initially, just enough to catch critical errors.
For instance, even in my Nginx reverse proxy settings on my own VPS, I use fail2ban patterns derived from my corporate experiences, but in a much simpler way:
# Nginx log format
log_format main_ext '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'$request_time $upstream_response_time $pipe $upstream_addr '
'$upstream_status $request_id';
# A simple regex pattern for fail2ban
# /etc/fail2ban/jail.d/nginx-badbots.conf
[nginx-badbots]
enabled = true
port = http,https
filter = nginx-badbots # My custom filter file
logpath = /var/log/nginx/access.log
maxretry = 3
findtime = 600
bantime = 3600
In the nginx-badbots filter I use for fail2ban, I don't use the complex User-Agent or URL patterns I saw in corporate networks, but rather simple regexes that only catch very obvious malicious bots. This allows me to use server resources more efficiently and reduces maintenance costs. When working on an AI production planning module in a manufacturing company's ERP, using prompt engineering and RAG patterns was inevitable. However, when it comes to AI application architecture in my own side projects, I initially start with a single provider like Gemini Flash or Groq, only adding complex structures like multi-provider fallback when truly needed.
My clear position is this: The knowledge and experience gained from the corporate world are invaluable. However, when transferring this experience to my own small projects, I must always consider the trade-offs and avoid the trap of over-engineering. Sometimes, saying "it's good enough" and proceeding with a simple solution proves much more efficient in the long run. In the next post, I will discuss the PostgreSQL partition strategies and vacuum monitoring issues I encountered while setting up the infrastructure for an anonymous Turkish data platform I developed.
