This article is a cautionary tale about my experience testing the Microsoft Sentinel data lake feature, which unexpectedly resulted in a ¥500,000 (approx. $3,300 USD) charge. First and foremost, I want to make it clear that this trouble was caused by my own lack of understanding and a series of misunderstandings.
To be honest, I debated whether to publish this, as it exposes my own foolishness to the world. As someone whose job involves specializing in Microsoft products, making a mistake like a cloud novice is quite embarrassing.
Furthermore, I worried this could create a misleading narrative about the long-awaited data lake feature. There are people on social media who criticize Microsoft for anything and everything, and I was concerned they might take my story out of context.
However, I decided to write this article to prevent others from facing a similar high charge due to the same misunderstanding. My apologies for the somewhat dramatic tone, but I strongly encourage anyone testing Microsoft Sentinel's data lake feature to read this to the end.
What Happened?
I was testing the Microsoft Sentinel data lake feature, which had recently become available in Public Preview.
https://zenn.dev/hirotomotaguchi/articles/202508_microsoft-sentinel-data-lake
In late August 2025, while checking my Azure billing, I discovered an unusually high charge of approximately ¥500,000 associated with my Sentinel resource.
My understanding was:
- I had only ingested about 90MB of data over 90 days.
- I hadn't performed any major operations.
- The data lake feature was in preview and should have included 30 days of free storage.
Given the circumstances, I couldn't fathom how a bill this large could be generated, so I immediately contacted Microsoft Support.
The Real Cause of the Bill
The Operation I Performed
I was conducting a test to "ingest data into the new Public Preview data lake and use it for analysis." Specifically, I was performing a restore operation from the Defender Portal > [Microsoft Sentinel] > [Data lake exploration] > [Search & restore].
My Misunderstandings
Herein lay my critical misunderstanding. I mistakenly believed the following:
- I thought it was the Data Lake's restore feature.
- Reality: Restore is not supported in the Data Lake.
- Reality: It was actually the "Search job" feature of the Azure Monitor service.
- I thought it was free because it was in preview.
- Reality: Search jobs and restore operations have separate charges.
- I thought it would be cheap because it was a small amount of data.
- Reality: Search jobs and restore operations have their own unique pricing models.
Technical Details
The Difference Between Data Lake and Azure Monitor
According to the explanation from Microsoft Support:
In Microsoft Sentinel Data Lake:
- The restore function is not supported.
- The search job item displayed under the Data Lake section is actually a feature of the Azure Monitor service.
In Azure Monitor Restore:
- It is billed under its own pricing model.
- A single execution is billed for a minimum of 2TB and 12 hours. This is because restoring data allocates additional compute resources to query the restored data, so a minimum restore data volume applies.[1]
- You are continuously billed for the restored data until you delete it.
- Restored tables have an
_RSTsuffix (e.g.,AADNonInteractiveUserSignInLogs_656_RST).
In other words, I was being charged the Azure Monitor restore fee of $0.10/GB/day × 2TB × number of days. No wonder it hit ¥500,000...
Looking closely now, the restore screen even has a "Note" with a similar warning... which I had completely skimmed over. This was entirely my fault.
This information was also available on Microsoft Learn.[2]
Long-term storage billing will be migrated to existing customer's data lake storage meters, but long-term stored data is initially only accessible via Microsoft Sentinel > Data Lake exploration > Search & Restore.
Lessons Learned and Countermeasures
1. Don't Be Misled by the UI and Always Read the Notes
Just because something appears within the Data Lake interface doesn't mean it's a Data Lake feature. In this case, an Azure Monitor feature was integrated, and although a note explained this, I overlooked it, leading to the unexpected charges. In hindsight, even though it was under a "Data Lake" tab, I should have recognized the UI looked similar to Azure Monitor and, more importantly, I should have read the note carefully.
2. Understand the Pricing Model Correctly
I was reminded that it's crucial to understand the pricing model before trying out a new feature. For this case, the relevant documents are:
3. Set Up Billing Alerts
Setting up alerts with a low threshold is a vital step to quickly detect unexpected costs.
4. Delete Unnecessary Resources
Delete any resources used for testing immediately after you're done. In this specific case, it's crucial to check for any tables with the _RST suffix and delete unnecessary restored tables right away.
A Request for Feedback
While this mistake was due to my lack of understanding, it's also true that the UI can be confusing. I've submitted feedback on this, and if you agree, I would appreciate your vote.
https://feedbackportal.microsoft.com/feedback/idea/1e8fe4f3-1994-f011-aa44-6045bd7f402a
In Conclusion
This experience taught me a painful lesson about the importance of caution when trying new cloud service features. The Microsoft Sentinel data lake feature itself is extremely useful, but it requires a correct understanding of the pricing models for its surrounding functionalities. I want to stress again that this was my fault, and I'm quite embarrassed. However, I hope this article helps others avoid making the same mistake.
Still, it's embarrassing to have been this mistaken, especially after writing a whole blog post about the feature... And that ¥500,000 bill really hurts...
Comments
Post a Comment