Refresh in Power BI: Understanding Incremental Refresh in the Service and the Role of Manual and Scheduled Refresh
Power BI offers multiple refresh mechanisms that serve different purposes. Manual and Scheduled Refresh control when data is refreshed, while Incremental Refresh controls how much data is refreshed. To clearly understand the difference, let’s walk through a single, consistent example using a sales dataset stored on OneDrive.
Manual Refresh vs Scheduled Refresh vs Incremental Refresh
Manual Refresh
Manual Refresh is used when an immediate or urgent update is required. It is triggered by clicking Refresh in Power BI Desktop or Refresh now in the Power BI Service. Each time a manual refresh runs, Power BI reloads the entire dataset, including both historical and recent data. This approach is useful for quick validations or unexpected data updates but is not practical for regular use.
Scheduled Refresh
Scheduled Refresh is Power BI’s automatic refresh mechanism. Instead of relying on users to manually refresh datasets, the Power BI Service refreshes data at predefined intervals such as daily or weekly.
Scheduled Refresh exists because, in real-world scenarios:
- New transactions are added regularly
- Source files (OneDrive, SharePoint, databases) are updated continuously
- Manual refresh is not scalable or reliable
With Scheduled Refresh enabled, reports remain up to date automatically without user intervention. Like manual refresh, Scheduled Refresh still reloads the entire dataset on every run. It defines when the refresh happens, not how much data is refreshed.
Incremental Refresh
Incremental Refresh defines what portion of data is refreshed. Instead of reloading the full dataset, Power BI refreshes only recent or changed data (for example, the last 30 days) while historical data remains untouched.
How Incremental Refresh Works
Power BI partitions the dataset based on a date column:
- Historical partitions are archived and not refreshed again
- Recent partitions are refreshed during each scheduled run
This logic is applied and managed by the Power BI Service, not by Power BI Desktop during normal refresh.
Manual Refresh in Power BI
Step-by-Step Explanation Using a Sales Data Example
To understand how Manual and Scheduled Refresh work in Power BI, let’s start with a simple and consistent sales data example. These steps focus only on traditional refresh behavior, where the entire dataset is refreshed each time.
Step 1: Sales Data Stored in Excel (Source Data)
The sales data is stored in an Excel file containing four columns: OrderID, OrderDate, Country, and Amount. The data spans multiple years, from 2022 to 2025, making it suitable for observing how Power BI handles full dataset refreshes over time.
Step 2: Connecting the Excel File from OneDrive
The Excel file is stored on OneDrive, and Power BI Desktop connects to it using a web URL, visible in Data Source Settings.
Because the file is cloud-based, the Power BI Service can access and refresh it directly after publishing, without requiring an on-premises data gateway. This setup is ideal for demonstrating both Manual and Scheduled Refresh.
Step 3: Initial Load in Power BI Desktop (Baseline State)
When the data is first loaded into Power BI Desktop:
- All historical records from 2022 to 2025 are imported
- No filters, parameters, or refresh rules are applied
- The dataset behaves like a standard Import model
This represents the baseline state before any refresh behavior is demonstrated.
Step 4: Changes Made in the Source File
Next, changes are made in the Excel file:
- New sales records are added
- Some existing records may be updated
This mirrors real-world scenarios where source data is continuously changing due to new transactions or corrections.
Step 5: Manual Refresh in Power BI Desktop
When Refresh is clicked manually in Power BI Desktop:
- Power BI reloads the entire dataset
- Both historical and newly added records are fetched again
- No distinction is made between old and recent data
Manual Refresh is typically used for immediate or urgent updates, such as validating newly added data.
Step 6: Publishing the Report to Power BI Service
Once the report is published from Power BI Desktop, it appears in the Power BI Service workspace as two separate artifacts: a Report and a Semantic Model (Dataset), as shown in the image above. From this point onward, all refresh activities are handled by the Power BI Service, not by Power BI Desktop. Any manual or scheduled refresh is executed against the semantic model in the Service, and Power BI Desktop is no longer involved in the refresh process.
Step 7: Manual Refresh in Power BI Service
When Refresh now is clicked in the Power BI Service:
A full dataset refresh is triggered. All historical and recent data is reloaded. The behavior is identical to a manual refresh in Desktop. This option is useful for one-time updates but is not scalable for large datasets.
Scheduled Refresh
Step 1: Configuring Scheduled Refresh
Scheduled Refresh is configured from:
- Workspace >> Semantic Model >> Settings >> Refresh
Here you define:
- Refresh frequency (daily / weekly)
- Time zone
- Failure notifications
Key Settings in Scheduled Refresh (What Each One Means)
1.Time Zone
This defines which time zone Power BI should follow when running the refresh.
- Example: UTC
- Important because refresh timing depends on business hours
- Especially critical for Incremental Refresh windows
2. Refresh Schedule (On / Off)
- ON >> Power BI refreshes automatically
- OFF >> Only manual refresh is possible
Turning this ON tells Power BI:
“You are allowed to refresh this dataset automatically.”
3. Refresh Frequency (Daily / Weekly)
This decides how often refresh should run.
- Daily >> Best for operational / transactional data
- Weekly >> Suitable for slow-changing data
This does not decide how much data refreshes. It only decides when refresh happens.
4. Notification Settings
Power BI can notify:
- Dataset owner
- Other email contacts
If refresh fails, alerts are sent automatically.
Step 2: Scheduled Refresh Behavior
When Scheduled Refresh runs, Power BI automatically refreshes the dataset at the configured time without any user intervention. During each scheduled refresh, the entire dataset is reloaded, including both historical and newly added data, regardless of whether older data has changed or not. This ensures reports remain up to date on a fixed schedule, but it does not optimize the volume of data being refreshed. In essence, Scheduled Refresh solves the “when to refresh” problem by automating refresh timing, but it does not address “how much data to refresh,” which becomes a limitation as datasets grow larger.
Incremental Refresh in Power BI
Incremental Refresh is designed to solve a common scalability problem in Power BI. As datasets grow over time, refreshing the entire dataset during every manual or scheduled refresh becomes inefficient. Most historical data rarely changes, yet traditional refresh methods reload it every time, increasing refresh duration, capacity consumption, and the risk of refresh failures.
Incremental Refresh works on a simple principle: historical data remains stable, while recent data continues to change. Instead of refreshing everything, Power BI archives older data and refreshes only the most recent portion of the dataset.
Step 1 : Preparing the Dataset for Incremental Refresh
The same sales dataset used earlier is reused, but now it is prepared for Incremental Refresh. At this stage, when the file is first loaded into Power BI Desktop, all historical data is visible from 2022 through 2025. Nothing is filtered yet, and Power BI behaves like a normal import model.
Step 2: Creating Date Parameters (RangeStart and RangeEnd)
Incremental Refresh in Power BI works using two special parameters: RangeStart and RangeEnd. These parameters act as placeholders that Power BI Service later replaces automatically during refresh.
In Power BI Desktop, these parameters are created from the Manage Parameters option.
Both parameters are defined as Date/Time types:
- RangeStart represents the lower boundary of data to load
- RangeEnd represents the upper boundary of data to load
At this stage, the values you assign (for example, 1 Jan 2023 and 1 Jan 2025) are only for design-time testing in Desktop. These values are not fixed and will be overridden by the Power BI Service during actual incremental refresh runs.
Step 4: Applying the Date Filter Using Parameters
1. The OrderDate column must be of type Date/DateTime, and it must be filtered using RangeStart and RangeEnd (OrderDate ≥ RangeStart and OrderDate ≤ RangeEnd) to enable Incremental Refresh.
2. Power BI Desktop may show only a subset of rows after applying the filter; this is expected, as the actual Incremental Refresh logic is applied in the Power BI Service, not in Desktop.
Step 5: Enabling Incremental Refresh on the Table
After preparing the query, Incremental Refresh is configured from the model view by right-clicking the table and selecting Incremental refresh.
Here, two important windows are defined:
Archive window :
This determines how much historical data is stored but not refreshed. In this example, data starting 2 years before the refresh date is archived. This means data from 2023 and 2024 is kept but frozen.
Incremental window :
This determines how much recent data is refreshed each time. In this case, only 1 year before the refresh date (2025) is refreshed repeatedly.
This configuration clearly separates old, stable data from new, changing data.
Step 6: What Actually Happens After Publishing to Power BI Service
This is the most misunderstood aspect of Incremental Refresh. Incremental Refresh does not run in Power BI Desktop, where every refresh is always a full refresh used only for development and testing. The real Incremental Refresh behavior begins after the dataset is published to the Power BI Service. From that point onward, Power BI applies the incremental logic whenever a refresh is triggered, either manually or through a scheduled refresh.
When a refresh runs in the Power BI Service:
- Data in the archived range (for example, 2023–2024) is not reloaded
- Only data in the incremental range (for example, 2025) is refreshed
- Power BI automatically manages data partitions internally, without exposing them to the user
What Happens When You Change Data in OneDrive and Refresh in Power BI Service (Incremental Refresh in Action)
After setting up Incremental Refresh and publishing the dataset to Power BI Service, the real behaviour becomes visible when the source data changes.
In this demo, the sales data is stored in an Excel file on OneDrive. Initially, the file contains data from 2022 to 2025. Incremental Refresh is configured to archive the last 2 years (2023–2024) and incrementally refresh only the last 1 year (2025).
Once the dataset is published, Power BI Service creates internal partitions based on this configuration. These partitions decide what data is frozen and what data is allowed to refresh.
Step-by-Step: Updating OneDrive Excel and Refreshing the Dataset
Step 1: Make Changes in OneDrive Excel
After publishing the dataset, new rows are added to the OneDrive Excel file:
- A new row is added for 2025 (for example, OrderID 1009 with date 25-May-2025)
- An existing row from 2024 is modified (for example, Country changed to Australia)
At this point, the Excel file clearly contains changes in both 2024 and 2025.
Step 2: Trigger Refresh in Power BI Service
Next, a refresh is triggered in Power BI Service by clicking Refresh now on the semantic model (dataset). In this example, we are triggering a manual refresh by clicking the Refresh button in the Service UI.
This refresh can be:
- Manual refresh (Refresh button clicked by a user), or
- Scheduled refresh (daily or weekly, configured in settings)
For Incremental Refresh, both manual and scheduled refresh behave the same way only the incremental data window is refreshed, while archived data remains unchanged.
What Power BI Service Actually Does During This Refresh
This is the key concept most people misunderstand. When the refresh starts, Power BI Service does NOT reload the entire Excel file again. Instead, it follows the Incremental Refresh rules that were defined during publishing.
Here is what happens internally:
- Archived partitions (2023–2024)
These partitions are frozen. Power BI does not re-read Excel rows for these years. Even if the Excel file has changes in 2024, they are ignored. - Incremental partition (2025)
This partition is actively refreshed. Power BI re-reads only the rows that fall within 2025. Any new rows or changes in 2025 are picked up.
What is Scheduled Refresh in Power BI? (Separate from Incremental Refresh)
Scheduled Refresh is Power BI’s automatic refresh mechanism that determines when Power BI should pull new data from the source. Instead of relying on someone to manually click the Refresh button every day, Scheduled Refresh allows the Power BI Service to update the dataset automatically at predefined intervals, such as daily or weekly, ensuring that reports stay up to date without manual intervention.
Why Scheduled Refresh Exists
In real projects, data keeps changing continuously:
- New sales happen
- New transactions are added
- Files in OneDrive or databases get updated
Manually refreshing datasets every day is not practical.
So Scheduled Refresh exists to:
- Keep reports up-to-date automatically
- Remove manual dependency
- Ensure business users always see fresh data
How to Configure Scheduled Refresh in Power BI Service
To configure Scheduled Refresh in Power BI Service, open the workspace and click the three dots (⋯) next to the semantic model (dataset). Select Settings, then navigate to the Refresh section. This is where refresh frequency, time zone, and automatic refresh schedules are defined for the dataset.
Key Settings in Scheduled Refresh (What Each One Means)
1.Time Zone
This defines which time zone Power BI should follow when running the refresh.
- Example: UTC
- Important because refresh timing depends on business hours
- Especially critical for Incremental Refresh windows
2. Refresh Schedule (On / Off)
- ON >> Power BI refreshes automatically
- OFF >> Only manual refresh is possible
Turning this ON tells Power BI:
“You are allowed to refresh this dataset automatically.”
3. Refresh Frequency (Daily / Weekly)
This decides how often refresh should run.
- Daily >> Best for operational / transactional data
- Weekly >> Suitable for slow-changing data
This does not decide how much data refreshes. It only decides when refresh happens.
4. Notification Settings
Power BI can notify:
- Dataset owner
- Other email contacts
If refresh fails, alerts are sent automatically.
Manual & Scheduled Refresh vs Incremental Refresh
1. Manual Refresh / Scheduled Refresh (Traditional Refresh)
What it is
Manual and scheduled refresh define when a Power BI dataset is refreshed. A manual refresh is triggered by a user clicking Refresh now in the Power BI Service, whereas a scheduled refresh runs automatically at a configured time (daily or weekly).
How it works
Each time a refresh is executed, Power BI reloads the entire dataset from the data source. This includes both historical and recent data, regardless of whether older data has changed or not. No data is treated as permanent or archived, which means every refresh performs a full data reload.
Benefits
- Easy to configure and understand
- Suitable for small datasets
- Ideal when historical data frequently changes
Limitations
- Becomes slow as data volume grows
- Consumes high memory and CPU resources
- Increased risk of refresh failures or timeouts
- Inefficient when older data is stable and unchanged
2. Incremental Refresh
What it is
Incremental Refresh defines what portion of data is refreshed, rather than when the refresh runs. It is designed to refresh only recent data, while keeping historical data unchanged.
How it works
Power BI divides the dataset into partitions based on a date column. Older data is stored as archived partitions, which are not refreshed again. Only the most recent time window (for example, the current year or last few months) is refreshed during each run. This logic is applied and managed internally by the Power BI Service, not Power BI Desktop.
Benefits
- Significantly faster refresh performance
- Reduced resource and capacity consumption
- Highly reliable for large datasets
- Scales efficiently to millions or billions of rows
Limitations
- Changes to historical data are not automatically reflected
- Initial setup is more complex (parameters and date filters required)
Conclusion
Manual and scheduled refresh determine when a dataset is refreshed, whereas Incremental Refresh determines how much data is refreshed during each run. For small datasets or scenarios where historical data changes frequently, traditional refresh methods are simple and sufficient. However, as data volumes grow and historical records become stable, full refreshes quickly turn into a performance and cost bottleneck. In such enterprise-scale scenarios, Incremental Refresh becomes essential to ensure faster refresh times, improved reliability, and efficient use of capacity. By refreshing only recent data and preserving historical partitions, Incremental Refresh enables Power BI models to scale confidently without sacrificing performance. In real-world enterprise reporting, Incremental Refresh is not just an optimization, it is a necessity.
