DusXproductions

create a Video Sequencer

Posted by DusX on 22 October 2011 | 2 Comments

Tags: Tutorial, Isadora, VJ, FREE

What is a video sequencer?
In our case it will be a module that is loosely modeled on a hardware drum machine. I will walk you through the building this software module in Isadora. At the end you will have a fully functional video sequencer / stepper that you can add to your own projects.

The internal switching unit of the sequencer will be our programming focus. This includes methods of passing multiple video streams (4 in our case, but in theory you can expand to as many as your hardware can support.) through to a cross-fader.
We will allow the source of each video channel to be defined, as well as; a global play and transition time for the clips.

This tutorial assumes that you are familiar with Isadora, and you have at minimum completed watching the video tutorials available.

To get started, download the Isadora fully functional demo or get the pre-release (latest release) if you wish to use the complete, fully functional, and FREE video sequencer provided at the end of this article.

This sequencer should work equally well on both Apple computers as PC's but I have not tested Mac so please let me know if you have any problems and we can try to work them out.

To get started we are going to create a few basic elements, in the following order:

1. 2 Channel Video Switcher
2. 2 Channel Video Cross-Fader
3. a simple user interface

The Video Switcher will become an actor (think object) of its own that will be  used within another actor that we will create called 'SequenceMixer'. All of the patching in this sequencer can be done without creating actors, but creating actors makes the interface cleaner and easier to move around. You can also reuse both the 'Video Switch' actor and the 'SequenceMixer' in other projects. In Isadora you drill into actors in very much the same way as you navigate into objects in Adobe Flash, so if you have any Flash experience it will help you here.

Step 1 build the Video Switcher.

A smooth video switch requires that the stream of video you are switching between be active before the switch occurs, and since we plan to add a cross-fade to the switch we know that whatever video is in the primary play position will need to remain visible to the mixer until at least the cross-fade is complete at which point the secondary video becomes the primary video. Also it is fair to look at this in the reverse, that the secondary video must be active before the cross-fade begins and remain this way until the next video (the third in this case) takes its place as primary.

We are going to use two 'selector' actors to accomplish this switching. We will always have two video streams feeding into our mixer so we are going to switch 4 videos separately in two channels.

Lets Do IT:

1. First add 4 'movie player' actors. Import a minimum of 4 videos and set each player to play a separate video.
2. Now add a 'selector' actor (found in the Control toolbox).
3. Set the inputs setting to 4
4. Connect each 'movie player' to one of the 'selector' inputs.
5. Repeat (2 thru 4)
6. Add 2 'projector' actors and connect the output of each 'selector' to the video in of one 'projector'. Make sure the projectors are set to additive and change the intensity to 50.

Recap:
You should now have each movie players video output connected to one input in each selector. Make sure the connections are in the same order. (if movie player 1 is connected to the first input in selector 1 then make sure it is also connected to the first input in selector 2)
NOTE: in my example I added 4 video delays (bypass set to on). They do nothing other than cleanup the connections.
You should also now be feeding the video out of the selector to a projector.
Download the completed Isadora file.

Step 2 make the selectors switchers

Currently your project plays only 1 video (if you see 2 you may want to double check the Recap at the end of the last section). Why? because each 'selector' actors select input is set to 1. Changing this setting from 1 thru 4 will select the corresponding values input (in our case a video stream, I suggest you test this). The next step as you have probably guessed is to automate the switching of these selector inputs. We need to feed a incremental value of 1 thru 4 to the select input of each selector. This value stream also needs to loop so that we finish each loop moving from 4 to 1. Additionally we need to offset one feed so that it is always 1 behind the other stream, giving us our primary and secondary video streams.

Lets Do IT:

1. add a 'counter' actor, set its Mode to wrap, minimum to 1 and maximum to 4. Wrap allows the counter to loop around from the maximum to the minimum. We set the minimum to 1 because the 'selector' actors value inputs start at 1, and the maximum is set to 4 since we only have 4 inputs.
2. add a 'pulse' actor, set its freq (frequency) to 0.33. Now connect its trigger output to the add input of your 'counter' actor. You should now see the counter incrementing along, 1 step per ~3 seconds (0.33 hz).
3. Now connect the output of your 'counter' actor to one of your 'selector' actors select input. This 'selector' actor is now your lead (or primary) video selector.
4. Your second 'selector' actor needs to select the video previously connected to your lead selector, so we will use a 'value delay line' actor with the default size setting of 1. Add it and connect the counter output to the 'value delay line' actors value input.
5. Finish the video flow by connecting the 'value delay line' actors output to the 'selector' actors select input.

If you got everything connected correctly you should now see the selectors cycling through the settings, and if you turn on your stage you should see video switching, although 2 videos will be composite on to each other making the switch a little unclear. So to make things make sense set the 'horz pos' (horizontal position) of one projector to 50 and the other to -50, now you should see have of each video beside the other. If you watch now you will see that one video will also play twice, first on one side then on the other, being replaced each time with the next video in the sequence.

Recap:
So what we did here might already make sense to you but it can't hurt to clear up a couple foggy spots. We will only added 3 elements in this step. First the counter with the wrap setting, so that we can count (eg:1,2,3,4,1,2,3,4,1,2...) then the pulse (remember that the speed of a pulse is set in Hz.. 1/cycles per second) to automate the incrementing of the counter, and finally the value delay along one value path to offset the value received by our secondary selector. The 'value delay line' actor has a default size of 1 set and this worked perfect for us since we only wanted a value passed 1 step behind the value being passed to our lead selector. If we change the size to 2 we will be activating the 3 value passed and always passing a value 2 steps behind the current counter position.. this would cause use a problem in this application, can you guess why? (hint: think about the cross-fader)
Download the completed Isadora file.

STEP 3 lets create our first user actor: Video Switch

We are now going to spend a few minutes to move most our our current functionality inside a custom actor (user actor). I talked about a few of the benefits of creating these actors earlier but one very important one for me right now is that I can no longer take screen captures of my entire patch.
Its easy and this part should move quickly.

Lets Do IT:

1. Add a 'user actor' actor. Right click its title bar (or menu: Actors->Rename Actor) and name the actor 'Video Switch'.
2. Select everything (I use ctrl 'a'), deselect the 'Video Switch' actor (I use shift and click), and now copy (ctrl 'c'). Deselect all by clicking the background. Double click the 'Video Switch' actor (a new tab will open named to match) and past it all in.
3. Now close the tab. Click the [X] next to the name 'Video Switch', a dialog will appear, select 'save and update all'. You have now created a new user actor.
4. We have some work to do though, our new actor has no inputs our outputs and can't be linked to anything. First since we are no longer inside our actor (not in a tab with the actors name) we can remove all of the logic that we want embedded in this new actor from this level of the interface. Select and delete every actor other than your new 'Video Switch' actor, your movie players and projectors. We will relink these to our 'Video Switch' actor soon.
5. Double click into your 'Video Switch' actor, and add a 'user input' (these actors are only available when inside a User actor). Place it near a 'Movie Player' actor, click the output cord of the movie player, remember where it connects to. Delete the 'Movie Player'. Now connect the output of the User input to the same locations the movie player was connected. (this is one reason I used the 'video delay' actors with bypass on before splitting my feed to the 2 selector actors). Do the same for each of the remaining movie players.
6. Add a 'user output' (again only available inside user actors). Delete one Projector and connect the selector output that was connected to the projector to the input of your new 'user output'. Do the same for the remaining projector.
7. OK, one more input to add. Delete your 'pulse' actor and add a 'user input' actor. Connect the out of the 'user input' actor to the add input of the 'counter' actor.
8. Double click each of your new user inputs and outputs and change the names. I named the inputs "Video1 input" etc.. and the outputs "Video1 output" etc.. The user input that links to the 'counter' actors add input doesn't need to be renamed since the auto-naming makes sense for now. Keeping these named clearly will make the actor easier to use now, and much easier later once you have forgotten the inner workings.
9. Click the 'X' to close the actor tab (or menu: Actors-> Save and Update User actor). A dialog will open, select "Save & Update All".
10. You should now see the top level of the Isadora patch (where you started) and it should look as it did before, with the addition of your new actor 'video switch'. Select each actor other than the new 'video switch', 'pulse', all 'movie player' actors, and 'projector' actors. Delete them. You should now only have these 8 actors in the patch.
11. Reconnect the 'movie player' actors to your new 'video switch' actor, try to keep the connections logical (video1 2 3 4 in order), and the outputs to your Projectors (remember Video1 output should be your lead video and 2 the secondary).
12. Reconnect the 'pulse' actor trigger out to your 'video switch' add input.

OK, so we just tore the path apart, wrapped one part of the functionality into a user actor, and then reconnected everything as it was using the new 'video switch' actor. With any luck you should now be able to run the patch and it will run identically to how it did at the end of step 2.

Recap:
User actors allow you to wrap program functions in re-useable blocks. You are required to define the inputs and outputs of your user actors. One complication in the process is that certain actors will need to be removed before the 'user inputs' and 'user outputs' can be linked. If multiple link threads were connected to these actors it can become confusing/difficult to reconnect the threads. I recommend adding additional actors between the beginnings and ends of the threads if there are multiple connections being made and you plan to convert the functionality into an actor (I used the video delay with bypass on after the movie players).
Download the completed Isadora file.

STEP 4 take control of the pulse time

Up until now we have had the pulse frequency set directly in the 'pulse' actor. We will now do a little work to allow us to edit this in a more easily readable/understood format, Seconds.

Lets Do IT:

 

1. add an 'absolute value' actor. (I use these often as xtra actors, to help with connecting one location to another... low overhead and good as long as you don't expect any negative numbers ;).
2. add a 'calculator' actor. Set the input operation to divide. Set input value 1 to 1.
3. Connect the output from the 'absolute value' actor to the 'calculator' actors value 2 input. (1 divided by the number of seconds gives you the frequency)
4. Connect the output from the 'calculator' actor to the freq (frequency) input of the 'pulse' actor.

Now if you change the input of the 'absolute value' actor (in seconds) the pulse will be set accordingly. Play with this a bit. Try very short periods like 0.25 and longer periods like 7.

Recap:
The 'pulse' actor uses Hz (cycles per second) as the standard setting of frequency.
You will find most generator type actors use Freq in Hz.
A simple calculation converts this value to a more human readable unit.
Download the completed Isadora file.

STEP 5 build on a video mixer

So this step doesn't include that many steps but it does give the sequencer an important final touch, allowing each image to cross-fade into the next. At the end of this step you will be able to adjust the transition time independently of the the play (pulse) time. If you set a transition time that is longer than the pulse time the cross-fade will not complete before the video switch is triggered (we won't disallow these settings as it is another effect that may be of use at times).

 

Lets Do IT:

1. Lets start by deleting one 'projector' actor. Doesn't matter which one since next you will delete the thread connecting the other.
2. Add a 'mixer' actor. Connect the 2 video outputs of your 'video switch' actor to the video in's of the 'mixer' actor (video1 to video1 etc..).
3. Connect the video output of the 'mixer' actor to the video input of the remaining 'projector' actor.
4. Now we just need to set the mix value input of the 'mixer' actor. To do this we are going to use a 'envelope generator' actor. Add an 'envelope generator' actor. We luck out.. the defaults work perfectly.
5. Link the output of the 'envelope generator' actor to the mix value input of the 'mixer' actor.
6. Link the trigger output of the 'pulse' actor to the trigger input of the 'envelope generator' actor.
7. Set the Horz pos input of your 'projector' actor to 0 (centered again)

You should now have the mixer transitioning from 0->100 over a period of 1 sec. You can now adjust the mixer transition time by adjusting the rate 1 input of the 'envelope generator'. Note this value is set in seconds already, so your 2 time settings now use a single time unit.

Recap:
We are now feeding the 2 video feeds from your 'video switch' actor into a 'mixer' actor. We are using the trigger from the pulse to do two things now: 1 it triggers the switching of the video stream in the 'video switch' actor, and 2 it starts the mixing of the video streams by triggering the envelope generator.
When a trigger is received the primary video is moved from video1 out to video2 out, but at the same time the envelope generator is triggered, this sets the transparency of video1 to 0, so that the primary video is still fully visible, and starts the transition of this setting to 100, where video2 is fully visible and video1 is hidden. When the pulse triggers again the process repeats. In the end we have a nice repeating 2 channel video cross-over.
A fun way to test the mixer now is to set the Play-Time (feeding into the pulse) to a higher number (say 10 seconds), and the Transition time (rate 1 on the envelope generator) to match. You should now see a constant slow cross-fade between the switching video feeds.
Download the completed Isadora file.

STEP 6 make it modular

We could leave it pretty much as is and just get to building an interface, but maybe you would like to have 2 video sequencers running at once? (I do;)
So we are going to take all of this functionality and wrap it up as another user actor, 'SequenceMixer'.

Lets Do IT:

1. Select all actors other than your thread beginnings/input and ends/outputs. So you should have your 'Calculator', 'Pulse Generator', 'Envelope Generator', 'Video Switch', and 'Mixer' actors selected. CUT.
2. Add a 'user actor', and double click into it.
3. Paste the the actors your Cut a second ago.
4. Add 'user input' actors, and 'user output' actors where actors used to be connected to these actors.
5. Rename the inputs and outputs (right click and select rename, or use the actors menu) so that they explain their use.
6. Close the actor tab, save and update all.
7. Relink the 'movie player' actors and the 'projector' actor.
8. Rename your new user actor to "SequenceMixer".
9. If you added an 'absolute value' actor previously (as I did) you can delete it. Since this value can be edit directly in the input of your new user actor 'SequenceMixer'.

I moved a little quicker this time and with less explanation since we have done this process before. Nothing new this time.. just showing you that user actors can be contained within other user actors. Your new actor should have 6 inputs (2 for times, and 4 video) and one video output.

Recap:
CUT and PASTE the functions you want to encapsulate into a user actor (object), add the inputs and outputs, and save. Relink things to get it working the same as before.
Download the completed Isadora file.

STEP 7 build an interface

I am not going to go into extreme detail here. Just a warning. We will add controls to select a video for each of our 4 video feeds, as well as sliders to control the transition time and clip play time.

Lets Do IT:

1. Set actor/control split to on.
2. Enable edit mode.
3. Add a 'bin picker' control to the control panel and drag it open to about 100px wide and 250px tall... or so.
4.  Drag the number (link ID) on the 'bin picker' control to the movie input of your first 'movie player' actor.
5. Double click you 'bin picker' and select check box "select all of selected type".
6. Copy your 'bin picker' control and past 3 copies. Move them so they are arranged nicely.
7. Drag their numbers to the inputs of the remaining 'movie player' actors.
8. Now add 2 sliders. Drag the link ID of each to one of the time inputs on the 'Sequence Mixer' actor.
9. Turn off edit mode.

OK, you now have control of the sequencer at a most basic level. If you are playing with your settings you will probably find that the time controls don't make setting useable times easy. Further the transition time is scaling strangely (in the decimal points).. these are little things I will leave up to you to figure out.. My finished file has them corrected, so if you like you can look through it to see what was done.

Here is an example of an interface for controlling the same video sequencer that has had a few more input/outputs used. Most are direct controls of the movie player, but one notable feature is the green bar that displays the current primary movie. This is accomplished bu setting an output in the 'video switch' actor for the current active selection, and then again passing out of the 'SequenceMixer' actor. The control is a horizontal radio button that listens to the output of this selection. You can link controls to the outputs of actors inside other actors, but I personally feel this is bad practice as it become difficult to locate links pretty quickly. Passing them through and out to the "surface" may take a little longer but is well worth the effort.

How to improve the design of this patch?
There is something you can do to make your actors more generic, thus more flexible for use in future projects. Try to rebuild the sequencer with a simplified 'Video Switch' actor. The simplified actor should switch only 1 channel of video. You will have to use two copies of the actor in the 'SequenceMixer' to feed your cross-fade system. Once you have this working I would suggest turning the cross-fade system into its own actor, this could again be a rather useful element for further projects.