New version for chunks

[JulStraus]

Problem statement

While we have in the current iteration the potential of including chunks, both based on length and based on the duration, there is in my opinion an interesting additional functionality which could be represented, chunks where the time period within one chunk is not present in another chunk.

Consider the following example:

ts = TwoLevel(1,1, SimpleTimes([1, 2, 3, 6, 1, 1, 1, 1, 1, 1]))
t_inv = first(strategic_periods(ts))

Utilizing chunk(t_inv, 3) on the system provides us with iterators containing the following periods

[sp1-t1, sp1-t2, sp1-t3]
[sp1-t2, sp1-t3, sp1-t4]
[sp1-t3, sp1-t4, sp1-t5]
[sp1-t4, sp1-t5, sp1-t6]
[sp1-t5, sp1-t6, sp1-t7]
[sp1-t6, sp1-t7, sp1-t8]
[sp1-t7, sp1-t8, sp1-t9]
[sp1-t8, sp1-t9, sp1-t10]
[sp1-t9, sp1-t10]
[sp1-t10]

while chunk_duration(t_inv,6) leads to:

[sp1-t1, sp1-t2, sp1-t3]
[sp1-t2, sp1-t3, sp1-t4]
[sp1-t3, sp1-t4]
[sp1-t4]
[sp1-t5, sp1-t6, sp1-t7, sp1-t8, sp1-t9, sp1-t10]
[sp1-t6, sp1-t7, sp1-t8, sp1-t9, sp1-t10]
[sp1-t7, sp1-t8, sp1-t9, sp1-t10]
[sp1-t8, sp1-t9, sp1-t10]
[sp1-t9, sp1-t10]
[sp1-t10]

I would be interested in non repetitive periods, i.e.,

[sp1-t1, sp1-t2, sp1-t3]
[sp1-t4, sp1-t5, sp1-t6]
[sp1-t7, sp1-t8, sp1-t9]
[sp1-t10]

and

[sp1-t1, sp1-t2, sp1-t3]
[sp1-t4]
[sp1-t5, sp1-t6, sp1-t7, sp1-t8, sp1-t9, sp1-t10]

This is in general a bit more tricky to utilize, as

1. the length of the iterator is not known beforehand, even if it could be deduced and
2. if the TimeStructure and the chosen number of periods (or their duration) is not consistent.

However, I still think it can be rather beneficial to include this also as it can be good for, e.g., demand that must be satisfied within a given period but the exact time is is satisfied is not important. Examples for this behavior is given by the PeriodDemandSink or CapacityCostLink.

Potential solution

In theory, this can be implemented quite fast, e.g., as

struct ChunkDurationTest{I}
    itr::I
    duration::TS.Duration
end
function Base.iterate(w::ChunkDurationTest, state = nothing)
    n = isnothing(state) ? iterate(w.itr) : iterate(w.itr, state)
    n === nothing && return n
    itr = w.itr
    next = TS.take_duration(isnothing(state) ? itr : Iterators.rest(itr, state...), w.duration)
    return next, n[2] + length(collect(next)) - 1
end

# With 
ChunkDurationTest(t_inv, 6)
# we receive
[sp1-t1, sp1-t2, sp1-t3]
[sp1-t4]
[sp1-t5, sp1-t6, sp1-t7, sp1-t8, sp1-t9, sp1-t10]

This is definitely not the most elegant approach, but it works. The onlydifference to ChunkDuration is related to the return of the next state as I take here as well the states which have passed already into account for defining the next state.

Is that fundamentally wrong through process for iterators? What are your thoughts @trulsf and @hellemo?

[JulStraus]

Additional thought: It would also be possible to consider varying durations for the individual chunks, although this makes it obviously a bit more difficult.

[trulsf]

I think it is a good idea - at least if we have the use cases. The first version is to some extent already covered by Iterators.partition:

ts = TwoLevel(2,1, SimpleTimes([1, 2, 3, 6, 1, 1, 1, 1, 1, 1]))
t_inv = first(strategic_periods(ts))

parts = collect(Iterators.partition(t_inv, 3))
# Returning
[sp1-t1, sp1-t2, sp1-t3]
[sp1-t4, sp1-t5, sp1-t6]
[sp1-t7, sp1-t8, sp1-t9]
[sp1-t10]

Note that partition is eager and returns the full vector of periods for each set in the partition. If we follow that convention, the implementation would avoid the length(collect(...)) in the state for a duration-based version:

struct PartitionDuration{I}
    itr::I
    duration::TimeStruct.Duration
end

partition_duration(itr, dur) = PartitionDuration(itr, dur)
Base.IteratorSize(::Type{<:PartitionDuration})   = Base.SizeUnknown()
Base.IteratorEltype(::Type{PartitionDuration{I}}) where {I} = Base.HasEltype()
Base.eltype(::Type{PartitionDuration{I}}) where {I} = Vector{eltype(I)}

struct IterationCutShort end

function Base.iterate(w::PartitionDuration, state = nothing)
    state isa IterationCutShort && return nothing
    y = isnothing(state) ? iterate(w.itr) : iterate(w.itr, state)
    isnothing(y) && return nothing
    chunk = eltype(w.itr)[]
    acc = zero(w.duration)
    while true
        push!(chunk, y[1])
        acc += TimeStruct.duration(y[1])
        acc >= w.duration && return chunk, y[2]
        nxt = iterate(w.itr, y[2])
        isnothing(nxt) && return chunk, IterationCutShort()
        y = nxt
    end
end

This will give similar results as above (although with vectors):

julia> pd = [p for p in partition_duration(t_inv, 6)]
3-element Vector{Vector{TimeStruct.OperationalPeriod{TimeStruct.SimplePeriod{Int64}}}}:
 [sp1-t1, sp1-t2, sp1-t3]
 [sp1-t4]
 [sp1-t5, sp1-t6, sp1-t7, sp1-t8, sp1-t9, sp1-t10]

[JulStraus]

I personally think it is fine that it returns a vector, although it might be beneficial to have a type for indexing variables over the partitions as it can make things easier. One approach could be:

struct PartitionDuration{T}
    part::Int
    per::Vector{T}
end

Base.iterate(pd::PartitionDuration, state = nothing) = iterate(pd.per, state)
Base.show(io::IO, pd::PartitionDuration) = print(io, "part-$(pd.part)")

For this, I adjusted the Base.iterate function a bit, including cleaning it up to avoid while true:

function Base.iterate(w::PartitionDurationIterator, state = (nothing, 1))
    isa(state[1], Iterators.IterationCutShort) && return nothing
    y = iterate(w.itr, state[1])
    isnothing(y) && return nothing
    part = state[2]
    chunk = eltype(w.itr)[]
    acc = zero(w.duration)
    while !isnothing(y)
        push!(chunk, y[1])
        acc += duration(y[1])
        acc >= w.duration && break
        y = iterate(w.itr, y[2])
    end
    isnothing(y) && return PartitionDuration(part, chunk), (Iterators.IterationCutShort(), part+1)
    return PartitionDuration(part, chunk), (y[2], part+1)
end

The aim of the adjustments was to reduce the number of exits from the function.

One thing which is still important is to decide what inforation should be included in PartitionDuration. It could be beneficial to include as well, if that is the case, the information of the TimePeriods.