@rniwa what is Apple doing here?

@smaug---- so I guess the problem is all the state exposed on History? If we did not expose that, would we have to expose it somehow on ShadowRoot? That each gets its own history? Presumably defaulting to no history at all for v1?

That kind of approach makes sense if you don't want to influence your container, but if you're a simple layout component that breaks expectations. Not sure if there's a good answer here. (We could make it depend on open vs closed...)

For open case I could possibly live with normal nested browsing context handling.
And for closed...no session history at all? I think that is the easiest for now. history.pushState/replaceState would do what? and what about history.go(x)? Throw?
I think throw is the only sane option, except perhaps for go(0)
How should the UI work? If shadow-iframe loads page X and then Y is loaded there, what should back button do? I guess nothing. That shouldn't be too bad. The shadow-iframe container component could manually call push/replaceState when needed, or shadow-iframe.src = "some new page"

Given the recent example, bug in Google Ads where their scripts caused the container page to go back, I'd prefer to never expose history state even in open case, but at least in closed case the state shouldn't be shared.

I would be okay with blocking history from "shadow-iframe" by default. And then later figuring out if we want to make that work. @hayatoito @rniwa?

I'm not sure if throwing would be best by the way. It depends on what kind of things end up being embedded. If you throw a bunch of existing pages will likely break when embedded in this way.

but if the page called go(<not_zero>), it sure would expect something to happen, and if we wouldn't throw, the page would be broken in some other way.
Or pushState(). After that one would expect history.state to be the thing passed to pushState.

This is tricky, but I'm not sure we have any other good option to effectively disable all history API in shadow-iframes.
Hmmm, crazy idea. Could we actually hide History interface and .history propery in the browsing contexts under shadow-iframes?

I would be okay with blocking history from "shadow-iframe" by default. And then later figuring out if we want to make that work. @hayatoito @rniwa?

I am okay with this. Blocking history from "shadow-iframe" by default should be a safe option.

[Updated] On the second thought, go() should work, IMO. Let me think further.

My opinion is:

• iframes in a shadow tree does not have any affect on window.history. Thus, states of iframes in a shadow trees is not exposed to window.history. In other words, window.history is owned exclusively by a document tree.
• go() should work as usual.

We might want to have another APIs, such as DocumentOrShadowRoot.history, if we want to make that work somehow later.

How can go() work if such <iframe>s don't have an effect on the history?

go() works in a history of a document tree. So I guess calling window.history.go() from a (developer of) shadow tree is a bad practice and does not bring a desired behavior in most cases.

To be a tad more precise, go() works in browsing context tree.

Not putting shadow dom iframe loads to session history, but having go() to work would lead to very surprising behavior. The browsing context in iframe could expect its page loads are in session history, but go() wouldn't actually know about those loads so it would always affect to some ancestor or sibling browsing context, never the browsing context in iframe.

(go() is super error prone API to use :/ )

Yeah, I think we can not have a perfect solution here. We have to compromise for window.history because it's globally accessible.

That doesn't really solve this issue though, does it? I guess we still have the corresponding issue against the HTML Standard to define what should happen here...

Let me reopen. Should we define how go() works?

Yeah, and all the other members.

So the conclusion here is to leave the history API work as is, even in closed shadow root,
and no blocking for shadow-iframes?
i.e. any <iframe> navigation history will be included in joint session history.

As dimitri's comment pointed out, whether shadow DOM is detectable is not a concern.

I'll work on a draft for pull request.

That does not seem the conclusion here... ;)

Let me explain the context:

I and @TakayoshiKochi discussed this topic yesterday, and we got a rough consensus:
"Let window.history API work as is. <iframe> in a shadow tree will contribute a top-level browsing context's joint session history."

That conclusion seems directly opposed to comments from folks from Apple and Mozilla that very much want encapsulation at all cost, even if it means losing features like document.currentScript.

Yeah, this is the opposite. Actually, I have changed my mind.

Yesterday, I tried to have a good proposal, but I can not have any concrete proposal if we are to block iframe in a shadow tree. That made me in a trouble. Some of the concerns in #184 (comment) can not be resolved.

After that, I am starting to tend to agree with Dimitir's comment in this particular case:

there is no big concern if In this perspective, exposing information or making Shadow DOM presence detectable is not as big of a concern.

Maybe I might change my mind again, but as of now, I am fine with "not blocking at all".

I'm very worried about exposing shadow iframes in the the light DOM's session history. It would make using history API in light DOM super error prone: it would become really hard for light DOM to know what the current state actually is.
History API is error prone enough even without shadow DOM. The recent incident where Google Ads started to do random history.go(-1) and forcing browser to go to the previous top level page (since go(-1) was called at wrong state or something) is just a hint of that.

@smaug---- is the concern shadow-dom specific?

I may not be understanding the case you quoted. In my understanding the injected Google Ads script mistakenly ran history.go(-1) which expected to navigate some <iframe> to go back to its history, but the history was empty and the main document went back, right?
This can easily happen if there are more than one injected <iframe>s and script tries to go back and forth independently for each <iframe>.

That's a reasonable concern but it should be addressed at nested browsing context and history API's behavior about joint session history, not at the level of shadow tree boundary.

Let's say a social network site starts using shadow DOM and an iframe for its own widget to display comments. Then, you don't want navigations within that iframe to affect the navigation history of the embedding page. It DOES happen today but I don't think that's really intended / desirable.

Yeah, that's one of concerns. However, I do not want Shadow DOM to control the behavior of iframes.
Shadow DOM should not have such a huge responsibility, IMO.

I prefer separating responsibilities to more primitives, in order to isolate our concerns.
For example, a new attribute for iframe, such as <iframe joinhistory=false>.

In general, that's component user's fault if they use a bad component.

As a general rule,

1. A component can do anything for outer trees. A component developer should have a responsibility to make a component a good citizen.
2. The opposite should not be true. An outer tree can not invade the inside of the component. That should be honored.

In this particular case, <ifarme> and history , this should be categorized to 1, I think.
This would not be a big concern, I hope.

This DOES seem like a pretty significant concern given changing this behavior later would not be backwards compatible.

I don't understand where your rule comes from. As far as I could recall, we've never reached a consensus on such a general principles (or even a consensus that having such a principle is a good idea).

The general rule that Hayato wrote in #184 (comment) isn't the matter of concern in this thread, and everyone doesn't have to agree on it at least here. He just expressed the reasoning behind our opinion.

BTW, I read Hayato's #184 (comment) wrong and I thought keeping the behavior as is was the conclusion of this thread in my previous #184 (comment). As everyone already pointed it out, it was totally opposite to what was changed in the spec change 08793be. Sorry for any confusion incurred by my message.

I don't understand where your rule comes from.

Yeah, that comes from my attempt to honer a well-established rule (I believe):

• Access from Inner To Outer: Allowed
• Access from Outer To Inner: Disallowed

We see a similar rule in most of programming languages. Thus, I think having this principle in our mind would be good for us.

Actually, you can see several instances of this rule in the current Shadow DOM:
e.g.

• a retargeting algorithm
• a concept of a unclosed node

The problem here is that exposing iframe history in the outer world breaks that very principle. Now the outer DOM can influence what gets shown in the iframe inside a shadow tree by using history.go(-1) and other history APIs.

Now the outer DOM can influence what gets shown in the iframe inside a shadow tree by using history.go(-1) and other history APIs.

Oh. That makes sense. This is a kind of "Outer to Inner".
However, given that history APIs is available via global window object, that sounds a historical technical debt and hard to avoid.

The problem here is that we do not have any concrete proposal other than "Not blocking at all".
Does someone have a clever idea?

We have options, I see about three paths:

1. Leak. Not a lot of work, has objections from Apple and Mozilla.
2. Don't make the iframe element work in shadow trees. Even less work, this is the status quo in the HTML Standard.
3. Design iframe elements differently inside shadow trees so they do not put properties on the global object and don't interfere with the history of the browsing context. Could be quite a bit of work, I haven't had time to investigate yet.

1. (4) throw in all the History attributes and methods when iframe is in shadow dom
2. (5) make attributes and methods of History no-ops
3. (6) hide .history property in shadow iframes.
4. (7) make .history property to be null in shadow iframes

(all these also need that loads in shadow iframes don't end up to session history)

(7) might not be too bad. Since it is not really acceptable for shadow iframe to affect to the global history, no pages using history API should be loaded to shadow iframes.

Or then (3), so that shadow iframe becomes a root of a session history itself. (but then, how would the UI work with several different session histories)

Thank you, @annevk , @smaug---- .

Blink is fine with 1 in #184 (comment).

Shadow DOM is originally designed to provide an encapsulation for DOM tree structures. It was not designed to provide an encapsulation for scripts. Thus, as long as nodes in a shadow tree are not leaked to outer trees, I am okay.

A behavior leakage (I am not sure how I can call this..) which would be caused by scripts is not what I expect Shadow DOM to prevent that. I do not want Shadow DOM to own such a role magically beyond its original role.

It would be nice that we could have a separate primitive which prevents these kinds of the behavior influences from happening, other than reusing a tree structure boundary provided by Shadow DOM.

A unit of a shadow tree and a unit of joint session history do not have to be always the same, I guess.

As I noted, all other implementers have a wider view as to what encapsulation means. So I don't think 1 is really an option at all, unless @rniwa and @smaug---- suddenly change their mind, which seems unlikely since they've been pretty consistent on this.

I see. Thanks. I think Blink can live in a world with option 3, given that DOM nodes are not leaked anyway. 2 is unacceptable, BTW.

However, I still prefer having new primitive by which we can explain the behavior of such a iframe.

e.g. a new attribute "XXX" for iframe.

On the top of it, we might want to explain the behavior of iframes in a shadow tree.

e.g. XXX attribute of iframe in a shadow tree is true by default.

That would make Shadow DOM less magical, hopefully.

(Looks like github has weird bug to change list numbering so I updated my comment a bit)

Let me extend @annevk's 3. How about this?

Let's define a new attribute "isolatehistory" (or something) for <iframe> and define the behavior for window.history as follows and make it usable for both <iframe> in document tree and in shadow tree. We might mandate the "isolatehisory" behavior by default (without attribute) inside shadow DOM.

• Bottomline:
  • navigation history inside <iframe> will not be merged to top-level browsing context's joint session history.
• window.history
  • <iframe isolatehistory> has its own independent session history.
  • go()/forward()/back() works on its own session history.
    • if the script really wants to go back for the whole document, window.parent.history.go(-1).
  • pushState()/replaceState()/state works on its its own session history.
  • length returns its own session history's length.
  • scrollRestoration affects only for the <iframe> itself.
• link anchor in <iframe>'s document
  • following a link will navigate inside <iframe>, and add it to its own session history.
  • target attribute on anchor works the same way without "isolatehistory" attribute.
• nested <iframe>
  • nested child <iframe>'s session history will be joined to its own session history, and will not leak to outer browsing context.
• browser's forward/back buttons (is this in any HTML spec?)
  • forward/back buttons follow the top-level browsing context's joint session history.
  • therefore inside <iframe>'s navigation history isn't counted. To compensate this, right-click menu can have back/forward menu item for the <iframe> (see next).
• context (right-click) menu
  • on "isolatehisotry"-iframe's region, context menu can include "back in frame" "forward in frame" menuitems to compensate browser's forward/back button not working for <iframe>'s content. (analogous to reload or view source)

This may still miss something (very probable). But could we converge on something like this?

That sounds fairly reasonable and similar to what I was thinking of. Could use input from @majido too here I think.

Looks good to me, basically. Thanks.

We might mandate the "isolatehisory" behavior by default (without attribute) inside shadow DOM.

I prefer iframe does not change the behavior by its root node.
Maybe, explicit is better than implicit here. I guess components developers are willing to a use new attribute explicitly if they want to have an encapsulation for iframes in their components.

I don't think we can allow the leaky behavior in shadow trees. Certainly not for closed shadow trees and probably not for open shadow trees either based on the discussion thus far. We could have an attribute to let non-shadow-tree iframes opt into this behavior, but for shadow trees I think it has to be mandatory.

I see. I think Blink can live in a such a world.

It sounds like that no one likes the current behavior of joint session history. Thus, we are changing the default behavior of iframes, with an exception for that being used in a document tree, due to the compatibility. That would make sense.

FWIW, I agree with anne's latest comment (#184 (comment))

And we are not changing default behavior of anything. Shadow DOM is special so shadow DOM gets special handling.

How does this attribute work, if the default is changed inside shadow DOM?

That is: I assume it has a reflected property, a boolean iframeEl.isolateHistory. Does this return false, since the attribute isn't there? Does it return true, since history is isolated? What does iframeEl.hasAttribute("isolatehistory") return inside the shadow DOM?

It seems very weird to flip the default behavior inside shadow trees.

@domenic I'm not sure we need an attribute, but if we do, it would be a no-op in shadow trees. (And we'd certainly not have magic like automatic attribute setting.)

A joint session history was a design mistake, right?

In other words, if we are allowed to design iframes from scratch, we never let iframes have such a behavior. Is my understanding correct?

I thought that there is a use case of a joint session history, but I do not see any use case in this thread.

If so, I am okay to re-design iframes for shadow trees. Forget joint session history at all. No new attribute. No customizable.

For the compatibility, we will have legacy iframes in a document tree, which have a joint session history.

Thanks @domenic for pointing it out.

One way is that rip out the "isolatehistory" attribute at all and enforce the "isolatehistory"
behavior under shadow DOM, and the legacy behavior in document automatically.

If someone come up with a serious need to become "isolatehistory" state in document,
maybe we can add makeHistoryIsolated() method to make the <iframe>'s history
isolated permanently? I guess there is no need to go back once isolated.

I'd like to add a clarification to my #184 (comment).

• when <iframe> is moved from a shadow tree to document tree, document tree to a shadow tree, or one shadow tree to another shadow tree (via appendChild() etc.)
  • remove all history entries when <iframe> is moved to a different node tree.
    (i.e. keep history object as is when <iframe> moves within the same node tree.)
  • if the destination is the document tree, after move is complete, <iframe>'s window.history will be the
    same state as document.appendChild(document.createElement('iframe')).contentWindow.history.
  • if the destination is a shadow tree, <iframe>'s window.history will contain one history entry of itself.

Yes, since moving iframe around DOM doesn't keep any of its history (even without any shadow DOM) anyhow.

@TakayoshiKochi are you interested in working out a patch for this for HTML?

Let me try, assuming no one objects to the proposals above. (was on a holiday week, will work on it next week)

Clarification:

Suppose the following tree:

#document
<!-- A -->
<iframe name="IF1">
  #document
  ... <!-- B -->
  <iframe name="IF3">#document ... <!-- D --> ...</iframe>
</iframe>
<iframe name="IF2">
  #document
  ... <!-- C -->
</iframe>

<shadow-host>
  #shadow X
  | <!-- E -->  
  | <iframe name="IF4">
  |   #document
  |   ... <!-- F -->
  |   <iframe name="IF5">#document ... <!-- H --> ...</iframe>
  | </iframe>
  | <iframe name="IF6">
  |   #document
  |   ... <!-- G -->
  |   <shadow-host2>
  |     #shadow Y
  |     | ... <!-- I -->
  |     | <iframe name="IF7">#document ... <!-- J --> ...</iframe>
  |   </shadow-host2>
  | </iframe>
  | <shadow-host3>
  |   #shadow Z
  |   | ... <!-- K -->
  |   | <iframe name="IF8">#document ... <!-- L --> ...</iframe>
  | </shadow-host3>
</shadow-host>

Any navigation in IF1, IF2, IF3 will be merged to window.history.
IF7 and IF8 have their own session histories.

As #184 (comment) navigation in IF5 will be merged to IF4.
Shall we also merge IF4 and IF6 (siblings)?
If they are merged, the notion of joint session history will be per node tree.

Then, if <a href="javascript:history.go(-1)"></a> link is put at location A, ..., L, it would traverse the history of

• #document for A, B, C, D, E, K
• #shadow X for F, G, H, I
• #shadow Y for J
• #shadow Z for L

It is still confusing at E, I, and K (outside <iframe> but inside shadow tree) but within each shadow tree scripts should be able to reach iframe's contentWindow.history anyway.

I don't understand the tree. Are the nested iframe elements in a new document created by their parent iframe element, or are they in the same document as their parent iframe element?

Yes, the nested iframe is in a hosting iframe's document, and loads its own document.
Sorry for the confusing notation.

It might help to repeat #document just as you repeated #shadow for nested instances. (I don't really have a strong opinion on the outcomes.)

Shall we also merge IF4 and IF6 (siblings)?

I think no. Any <iframe> in shadow DOM should get its own session history tree to keep the setup simpler. There after all isn't top level browsing context inside shadow DOM, like there is in light DOM: the browsing contexts in light DOM form a tree, browsing contexts within a shadow DOM may not form a tree.

@smaug---- thanks for the comment (btw the comment seems having broken formatting, could you fix so I won't miss anything? - thanks for fixing!).

I didn't have strong opinion about whether or not to merge IF4 and IF6. Probably it is nicer not to merge those as they don't share history with their parent.

I have been doing feasibility study and getting feedback from colleagues, and
it turned out that having a separate history list for each frame in shadow DOM
in Chrome/Blink is harder than I originally thought, and the consistency of browser
UI back/forward button for non-shadow <iframe> and shadow <iframe> navigation
made me turn to consider @smaug----'s #184 (comment) again.

In the next comment I'll illustrate my current thinking of how browser UI's back/forward buttons
and window.history.go() work (warning: long).

One of the feedback for showing 'back/forward in this frame' in context menu is
that it is hardly discoverable, as <iframe>s are not obviously visible to the user,
not to mention whether they are in Shadow or not.
The same could be said to "view source in this frame" etc., but probably back/forward
has more usage.

So I scratched the idea and started looking at how browser's back/forward buttons
interact with navigation in <iframe>s.

See this demo (Shadow DOM not included):
https://takayoshikochi.github.io/tests/005_iframes.html

The startpage (A1) has a link to next page (A2), and A2 has 2 <iframe>s,
the one loads B1 and the other loads C1. B1 and C1 has a link to B2 and C2.

As I understand browsers create a history entry for a snapshot of a page including
state of frames. Let's assume you go to A2 and then navigate C1 to C2, then B1 to B2.
Then browser's back/forward history list contains entries like:

Entry1: A1
Entry2: A2 + B1 + C1
Entry3: A2 + B1 + C2
Entry4: A2 + B2 + C2

Within each frame in the demo, it has 2 buttons (back/forward), which directly calls
window.history.back()/window.history.forward(). Clicking on any of the 3 "back"
button will navigate to the previous entry in this demo (confirmed on Safari/Firefox/
Chrome on Mac). Clicking on browser UI "back" button behaves the same.

Next, let's assume those iframe B and C are enclosed in Shadow DOM, and
assume what I proposed previously (#184 (comment) -
separate history list for each <iframe>, history.back() traverses its own history,
shadow <iframe>s do not participate in top-level joint session history).

There are choices how browser UI's back/forward buttons and each frame document's
back/forward buttons.

Question 1: what do browser UI's back/forward buttons work like?

If it should be consistent withhistory.back()/history.forward() in the outer frame A,
clicking on UI back button at Entry4 state above will directly go back to page A1 (Entry1).
Or do we keep the same behavior without Shadow DOM, and go back to Entry3 state?
(but history.back() would go back to A1 anyway, because session histories of B and C are
not in the joint session history).

Question 2: If the answer to the previous question is the latter (UI back button can navigate
history in shadow <iframe>s), and if history.back() called in the frame C at Entry4 state
and frame C goes back to C1 from C2.
What should the browser UI back/forward button work?

Possible answer A: a newentry is created with the new state (Entry5) - then
clicking UI back button click will end up going back to Entry4??

Entry1: A1
Entry2: A2 + B1 + C1
Entry3: A2 + B1 + C2
Entry4: A2 + B2 + C2
Entry5: A2 + B2 + C1

Possible answer B: the new history state replaces Entry3. Probably this is broken.

Entry1: A1
Entry2: A2 + B1 + C1
Entry3': A2 + B2 + C1
Entry4: A2 + B2 + C2

Other answers?

shadow iframe should have its own session history, and that could be exposed only to web page using history API. User doesn't know what is iframe and what is just some random element which content is changed dynamically. So there is no need to have any UI for the shadow iframe history.

I think the point is:

e.g.

1. User visits A1.
2. User clicks a link in A1 => User visits A2, which contains shadow iframe B1 and shadow iframe C1
3. User clicks a link in B1 => User sees: A2 + B2 + C1
4. User clicks a link in B2 => User sees: A2 + B3 + C1
5. User clicks a link in C1 => User sees: A2 + B3 + C2
6. User clicks a back button in UA's toolbar => Back to state 1 (A1), rather than state 4 (A2 + B3 + C1), because we do not keep a joint-history anywhere.

As far as I understand, this is the current proposal, right?

From user's perspective, 6 looks a weird movement.
Users would feel that one click of a back button rewinded 4 history items, rather than one history item.

Yeah, the point is that navigation within <iframe> works regardless of Shadow DOM,
but browser UI back/forward button works differently after iframe navigation, whether
it is in Shadow DOM or not. Is this difference acceptable?

This is slightly diverged from the original topic (window.history API) and the spec only says
in non-normative section:
https://html.spec.whatwg.org/multipage/browsers.html#history-notes
So, even when history API's go()/back()/forward() follow its own history list, browser UI can still
follow the whole joint session history, which is implementation specific.

To amend what hayato said(#184 (comment)), we can keep a joint session history somewhere, and browser can use it for
back/forward buttons. We don't expose the joint session history to any window.history
interface.

We used to have (implicit?) assumption that history objects in main page/iframes has
history list synced with browser UI's back/forward button,
but in the new world, any history.go() in each iframe in shadow DOM is local,
and such local movement doesn't necessarily matches the old world's synced,
linear movement, and there can be multiple possibility of how browser's back/forward
button works.

@smaug----, in my #184 (comment)
I had a mental model that user "clicked" some link within an iframe and navigated to the
new content in the iframe. Reading your comment

what is just some random element which content is changed dynamically

I am not sure you got that point.

Usually a user who clicks on a link would expect browser's back button would bring him/her
back to the previous state, I guess.

To pose the problem in a flipped way, after the following user-initiated (click on link) navigation
happend in the demo, assuming
each iframe were in the Shadow DOM.

Entry1: A1
Entry2: A2 + B1 + C1
Entry3: A2 + B1 + C2
Entry4: A2 + B2 + C2

Each history object has the following list:

A: [A1, A2] current index = 1
B: [B1, B2] current index = 1
C: [C1, C2] current index = 1

then history.back() is executed in frame C, each history object looks like:

A: [A1, A2] current index = 1
B: [B1, B2] current index = 1
C: [C1, C2] current index = 0 (showing C1 content)

What will happen for each frame and each history object if browser UI's back/forward button
is pressed?
(again, this may be out of scope for the HTML spec, but everyone here has to implement something for each browser :)

In summary, here are options:
3. <iframe> in Shadow DOM doesn't participate in parent's session history (no joint)
3.1 such <iframe> has its own session history list. history.go()/back()/forward() follows the list.
3.2 such <iframe> doesn't have its own session history list. history.go()/back()/forward() won't work, except history.go(0). (probably pushState()/replaceState() won't, either)

3.1:
pro: from the scripting perspective, it cleanly separates the session history, no leak.
con: breaks browser's back/forward buttons in a weird way once navigation is done in shadow iframe via History API (if browser implementation still keeps the joint session history from the whole frame tree and use it for back/forward navigation, as described in my previous comments)

3.2:
pro: can maintain browser back/forward button behvaior, as usage of any history API won't navigate
con: embedded page doesn't know whether history API restriction exists, and some may be broken.

One solution for the con of 3.1 could be that browser UI back/forward buttons don't follow the internal joint session history from the whole frame tree, but use the joint session history of top-level browsing context in the new world (i.e. session history exposed to the top-level window.history).
Quoting #184 (comment) from @smaug----

what should back button do? I guess nothing. That shouldn't be too bad.

But this is with the downside that the browser back/forward behavior is different from
when <iframe> is used in light DOM.

Implementation feedback for 3.1 (specific to Chrome/Blink) is that lots of current code assumes the joint session history and the top-level page has 1:1 mapping, quite a lot of rework has to be done to separate the list, and UX breakage of browser back/forward buttons is anticipated.

Well, shadow DOM is a different beast, effectively trying to hide some implementation details of random elements, which is why I think it is totally fine that the session history API inside shadow iframes is just an API thing, not affecting UI. And the API there is exposed anyhow largely to keep existing pages which use history API working even inside shadow iframes.

3.1, if implemented so that shadow iframe's session history doesn't show up in the UI, should be quite easy to implement in Gecko, but if we need to somehow combine the light and various shadow histories, the implementation, and UI, becomes a lot harder to implement.

@smaug---- thanks for the feedback. we are also concerned about compatibility of existing pages embedded in shadow iframes. So 3.2 is not a viable option for us, though with 3.1 some implementation difficulties are foreseen as you wrote.

@rniwa any opinions or ideas?

@travisleithead anything from Microsoft?

Let me try to move forward - we seem to get stuck, no perfect idea to solve all the constraints and concerns so far.

As we illustrated in #184 (comment) and #184 (comment), if we had separated joint session history to document and shadow iframes, it is expected that there would be inevitable browser back/forward button behavior inconsistency against user expectation.

If anyone cannot come up with a way to make it compatible with how back/forward button behaves on current browsers, we cannot mandate the separate list idea in V1.

As in @annevk's #184 (comment)

That kind of approach makes sense if you don't want to influence your container, but if you're a simple layout component that breaks expectations.

there's much possibility that current behavior is anticipated.

Any wisdom?
@rniwa @smaug---- @annevk @travisleithead @domenic

If anyone cannot come up with a way to make it compatible with how back/forward button behaves on current browsers, we cannot mandate the separate list idea in V1.

Why such limitation? Why does the shadow iframe need to show up in the browser UI?
One could also consider (conceptually) navigation in shadow iframes as replacing iframe with a new iframe[1]. That ends up removing the previous entries from session history and just add the new current one. Remember, user doesn't know what an iframe is. One way to consider this all is that, effectively session history is top level page navigations and state changes inside those which user can back<->forward. Iframe navigations are states within the top level page, not much different to pushState. And shadow iframes don't add such states.
([1] Note, webkit/blink are broken when doing dynamic changes. In some cases they end up loading session history entries to browsing contexts which never had such url loaded.
http://mozilla.pettay.fi/moztests/history2/Start.html is an old test for that.)

However if we do want to expose shadow iframe navigations in UI, UI could merge the various session history trees by looking at the creation time of the entries. That is anyhow the approach the HTML spec has taken for the normal DOM.
So, back button would end up checking which session history tree has the previous entry created most recently and navigate that tree.
Whether the shadow iframes should be exposed in the UI should be spec'ed.
Now that I think this some more, this shouldn't be too hard to implement in Gecko.
Session history transaction list would just contain entries for various session histories. And if history.go(...) was used, that would need to skip entries not for that particular session history.
UI would operate on the transaction list in the normal way.

One problem with merging various session history trees in UI is that top level page's history.back()/forward()/go() wouldn't map to it anymore. I guess that is fine.

Why such limitation? Why does the shadow iframe need to show up in the browser UI?

The rationale for us is that "it works that way without shadow, why do we limit iframe history
would not show up in the browser UI?".

I'd say that having a separate history list for each iframe is a saner design
if we could do from scratch, as you also said, e.g. history.go() is very error prone API.
So I tried to have a working proposal, but failed to come up with a solution to provide
consistent behavior when history API in iframe (which has separate history list)
and browser global back/forward button are used.

For us, not including any shadow iframe history in browser UI back/forward button at all is not an option.

Now that I think this some more, this shouldn't be too hard to implement in Gecko.
Session history transaction list would just contain entries for various session histories. And if history.go(...) was used, that would need to skip entries not for that particular session history.
UI would operate on the transaction list in the normal way.

I thought a similar solution, and but once history API is used in shadow iframes and UI back button is used, it easily went to weird state or weird behavior, and we haven't come up with a clear solution.
If one of the two is exclusively used, it can be very consistent, but both are mixed, it can go some
weird state in a few steps.

Could you have a more formal proposal/algorithm how it sanely works?

I played a little bit with iframe but it looks like all browsers merge session history even for cross-origin navigations. Given that we're not certain special casing session history for iframe inside a shadow tree makes much sense but we don't have a strong opinion either way.

We're curious what @travisleithead and the rest of Microsoft folks think of this matter.

I am now checking the status of all Shadow DOM v1 spec issues.
It looks that there is no interoperability risk if we do not have any special handling for session history for iframe in a shadow tree.

I would like to send an "Intent to ship: Shadow DOM v1" in Blink next week or later.
Regarding iframes in a shadow tree, if someone still think this is an important issue and there is a great risk for interoperability, please let me know that.

Why wouldn't this be an important issue?
I'm still rather strongly against using the same session history for shadow iframes, since it causes really unexpected behavior.

And of course there isn't interoperability risk atm if there are no browsers shipping Shadow DOMv1, we can ship whatever random stuff at that point. The thing is that we need to get this kind of core issues sorted out before anyone ships, otherwise we'll end up to the same mess we have/had with previous Shadow DOM version. (Is it v0)

@smaug---- Thank you for the feedback.

@TakayoshiKochi, @rniwa, @smaug----, @travisleithead
Could you discuss the pros and cons? We need to have a consensus.
As far as I can read, only Mozilla is still strongly against using the same session history for shadow iframes.
Other browser vendors do not have a strong opinion on this, I think.

I could rephrase that a bit. s/Mozilla/smaug/ and a question is here that why would we be ok to break encapsulation here when we're trying to achieve it elsewhere?

And functionally it is very surprising if the light dom can affect to the state of shadow iframes via history API, and even more surprising is if different shadow trees can affect to each others via history API.

We understand your concern.

The point we'd argue is that "history API usages cause unexpected behavior" (current behavior) vs "splitting session history for shadow iframes causes unexpected behavior for browser UI back/forward navigation".

As I understand, no one has come up with a clear solution to the question posed in #184 (comment).
(I tried in several comments but failed, which is a shame)

In @smaug---- 's #184 (comment)

Session history transaction list would just contain entries for various session histories. And if history.go(...) was used, that would need to skip entries not for that particular session history.
UI would operate on the transaction list in the normal way.

"UI would operate on the transaction list" is the tricky part.
For example, when an iframe did history.go(-1), it would create a new transaction.
Then UI back button would forward the iframe?

I don't think there are infinite options how UI back button would behave in this case,
but there are only a few possible ways. As far as we explored for Blink, they were either
too complex to implement (if not impossible) or more confusing behavior to users.

The most plausible idea I came up with was that such separated session history for
shadow iframes would not be added to the list for UI back/forward button. This turned
out to be implementable, but I got comments internally that users would get confused.

So we are comparing cost/benefit of implementing the session history split vs
keeping the current behavior.

Keeping the current behavior, basically no cost, but has negative benefit that you already
mentioned several times. Pro is that there is no change in iframe and history API behavior,
in a sense.

On the other hand, splitting the session history, we foresee some non-ignorable implementation
cost (plus cost for inventing/defining a reasonable behavior), clear benefit for encapsulation,
and uncertain con for unexpected inconsistency for UI back/forward button behavior.

We are in the pragmatic position to favor the former.

One problem with merging various session history trees in UI is that top level page's history.back()/forward()/go() wouldn't map to it anymore. I guess that is fine.

I'd agree that UI back/forward list and top-level session can diverge, but this is not the
point we are against.

The most plausible idea I came up with was that such separated session history for
shadow iframes would not be added to the list for UI back/forward button.

This is what I'd do. Users don't know what an "iframe" is. and iframe could be implement by the page using just random div element which doesn't add anything to the session history.

This turned
out to be implementable, but I got comments internally that users would get confused.

Why? If user doesn't even know there is an iframe to navigate back<->forward, what is there to get confused?

I would, at least initially, make shadow iframes to have separate session history so that we don't break existing pages using session history, yet don't leak information about shadow iframes to other shadow iframes or light DOM and not expose shadow history in UI.
Then, if we later figure out that shadow iframe history needs to be exposed to the UI, we try to come up with some solution.
That is my pragmatic proposal for this issue.

(In my mind not fixing session history for shadow DOM case is similar to the v0 shadow DOM where is-in-document issue was totally not figured out at all.)

This turned
out to be implementable, but I got comments internally that users would get confused.

Why? If user doesn't even know there is an iframe to navigate back<->forward, what is there to get confused?

Suppose there are 2 components, one with Shadow DOM and the other without Shadow DOM
otherwise identical. Even when history API is used only within <iframe> in the components,
once history API is used, browser UI back/forward button behaves differently if shadow-iframe
history isn't exposed to UI. Users cannot distinguish them from what they appear. Therefore
it gets confusing.

I don't have the backing data to support this, such as how frequently this pattern would be used in real world, unfortunately. But I guess the situation happens for polyfilled Shadow DOM and native Shadow DOM for the time being until everyone implements.

Having separate list and not exposing history to UI doesn't come as free. Even when we can
ignore the cost for implementation, users may have to pay the cost for incompatible behavior,
in turn for developer (history API user, component developer)'s convenience and sanity.

So the question is whether we (browser implementors) can impose the (possible) pain to the users,
or the benefit of the separation wins against the pain.

I haven't read too far past #184 (comment), but for number 3 to be possible, which is

Design iframe elements differently inside shadow trees so they do not put properties on the global object and don't interfere with the history of the browsing context. Could be quite a bit of work, I haven't had time to investigate yet.

then this means that Nodes that are distributed into a ShadowDOM tree (into slot elements) must somehow be aware that they have been distributed into a ShadowDOM tree (even if the tree is closed) in order to decide to behave differently than when not in a ShadowDOM tree. There is currently no part of webcomponents spec (as far as I'm aware, please let me know if otherwise) that would explain how a Node can possibly be aware of such a thing as having been distributed into a shadow tree.

So, this leads me to link here to an idea I made earlier: distributedCallback. The slot argument to distributedCallback is null when the shadow tree that the element is distributed into is closed. This method could possibly be the explanation as to how an <iframe> can possibly know to behave differently inside a shadow tree, if that route is taken with iframes.

Possible answer B: the new history state replaces Entry3.

Entry1: A1
Entry2: A2 + B1 + C1
Entry3': A2 + B2 + C1
Entry4: A2 + B2 + C2

I'm leaning to this. (I got the same idea and then realized that perhaps it was proposed already)
The replacement happens only when history.back()/go(<0) is explicitly used in shadow browsing contexts. If that doesn't happen, the UI works just the way it works now, since there is the session history transaction list of session history entry trees

Thanks @smaug---- for giving more thought on this.

How feasible do you feel for implementing it in Gecko/Firefox?
I guess it is implementable in Chromium/Blink, but with much effort - it needs huge refactoring;)

I'm curious what do you think if you get back to Entry3' state now, and if iframe B navigates B3,
a new entry (Entry3'' = A2 + B3 + C1) will be created. Usually going back the history and make a new
navigation, will prune the old forward history, in this case Entry4, which means we lose "C2".
But still in iframe C's history list, C2 exists, so history.forward() in C could navigate to C2.
Do you think you allow UI forward button to make C1->C2 navigation (with keeping B3 in
iframe B) in this case?

Entry1: A1
Entry2: A2 + B1 + C1
Entry3': A2 + B2 + C1
Entry3'': A2 + B3 + C1 <== new entry after Entry3'
Entry4: A2 + B2 + C2 <== ???

I am now triaging issues. Is there any update on this issue?

@smaug---- any update on this?

Sorry, no. I'm hoping this can be chatted during TPAC. But I'll try to look at this before that anyhow.

Thanks for the update. I'll be at TPAC next week.
Let's chat about this.