Skip to main content

Power BI

New

Incremental Refresh using custom date format

Vote (11) Share
Ahmad Anuar's profile image

Ahmad Anuar on 15 Nov 2018 12:01:28

Please enable custom date format in incremental refresh of dataflow just like in Power BI desktop features.

Comments (1)
Ahmad Anuar's profile image Profile Picture

5419777e dfcf-ea11-a812-000d3a579c38 on 11 Aug 2020 16:28:50

RE: Incremental Refresh using custom date format

To bad I can't give more than one vote for this!

The support for incremental refresh using PBI Dataflows are just horrible. If you have a datetime column in your data source it's ok, but if you just have a DW date key it's horrible. In my case I have a YYYYMM key in the data source, but it's like as bad if you have a YYYYMMDD key. The problem is to re-format the YYYYMMDD key to datetime while keeping query folding.
The workaround is to first create some fake datetime column, maybe using:
Table.AddColumn(source, "Fake", each DateTime.FixedLocalNow(), type datetime)
This will make it possible to configure incremental refresh.
Once this is done you can edit the entity and re-write the automatically populated filter, in my case I used the recommended method described here: https://docs.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh#rangestart-and-rangeend-parameters.
So what I did was to re-write the automated filter from:
Table.SelectRows(#"Added Fake date column", each DateTime.From([Fake]) >= RangeStart and DateTime.From([Fake]) < RangeEnd)
To:
Table.SelectRows(#"Added Fake date column", each [DateKey] >= dateToYYYYMM(RangeStart) and [DateKey] < dateToYYYYMM(RangeEnd))
Where dateToYYYYMM = (x as datetime) => Date.Year(x)*10000 + Date.Year(x)*100

But doint this, everytime I update the incremental refresh parameters, I need to re-write the filter.

It looks to me that MS has done a pretty good implementation for datasets, but totally forgotten to do the same for Power BI Dataflows