• This is a collection of assorted resources - not a complete, step-by-step system!
• Free for personal use (if you share please attribute me).
• The key is to explore & internalise the ideas before applying them to your instrument.
I believe that attention to structure in improvisation is imperative - and sadly neglected. At least it is hardly ever discussed. And there so many great examples all around us in all aspects of the arts - even the most banal Hollywood blockbuster is incredibly fine-tuned in terms of structure, and its intended effect on the viewer.
Improvising musicians have many tools at their disposal, many options in any one moment and how we choose to deploy them should be determined by the stories we are telling. Yes, stories - plural. Just as in novel-writing and film, smaller stories are framed inside larger ones, often with multiple layers of time co-existing in our imaginations at the same time. The same applies to a musical performance.
That is, we can have the story of an individual solo; a succession of short stories if there are multiple solos; the journey of the entire song; the story arc of a sequence of songs (say a set), each still with all its micro-elements; and, the most important of all for the listener (even if they are aware of this only subconsciously), the narrative arc of an entire concert.
Ok, we don't live in an ideal world where we can directly influence all of these things all of the time, but we can choose to consider it a priority. And in so doing we can influence those around us. Your playing will be deeper and more fulfilling if you move your attention from "this note, this chord" to considering the shape of the music that is just out there in the future when, instead of asking "Where am I?", you ask "Where am I going?"
On a practical note I have found transcribing entire arrangements very enlightening in this regard (I refer here to the entire structure, not to every individual part). It is a way of considering the narrative journey of a song: where the composer lingers, surprises, returns - or doesn't, where there is repose, tension, movement, textural change, dynamic change. These are the kind of elements that create the emotional journey of the piece.
Here are a few songs of various styles I transcribed with this in mind. I suggest you do the same (transcribe a few songs in various styles) then make tidy charts. One consequence is that you'll have to make critical decisions about which are the key elements of the song that simply must be on the leadsheet. Thus you will have to carefully consider the individual worth of the component parts in terms of their importance to the whole.