Génération de Vidéo

ÉVOLUTION DE LA GÉNÉRATION VIDÉO :

2022    2023           2024              2025
  │       │              │                 │
  ▼       ▼              ▼                 ▼
┌─────┐ ┌─────┐      ┌─────────┐      ┌─────────┐
│Make-│ │Gen-2│      │  Sora   │      │  Sora   │
│A-   │ │     │      │ Preview │      │  Public │
│Video│ │     │      │         │      │+ Runway │
└─────┘ └─────┘      └─────────┘      │  Gen-3  │
  │       │              │            └─────────┘
  │       │              │                 │
Quelques  ~4 sec      ~60 sec          Minutes
frames   256px       1080p HD         haute qualité

from openai import OpenAI

client = OpenAI()

def generate_video(prompt: str, duration: int = 10) -> str:
    """Génère une vidéo avec Sora."""
    response = client.videos.generate(
        model="sora",
        prompt=prompt,
        duration=duration,  # en secondes
        resolution="1080p",
        aspect_ratio="16:9"
    )

    return response.url

# Exemple de prompt détaillé
prompt = """
A stylish woman walks down a Tokyo street filled with warm glowing neon
and animated city signage. She wears a black leather jacket, a long red dress,
and black boots, and carries a black purse. She wears sunglasses and red lipstick.
She walks confidently and casually. The street is damp and reflective,
creating a mirror effect of the colorful lights. Many pedestrians walk about.
"""

video_url = generate_video(prompt, duration=15)

import requests

class RunwayAPI:
    def __init__(self, api_key: str):
        self.api_key = api_key
        self.base_url = "https://api.runwayml.com/v1"

    def generate_video(
        self,
        prompt: str,
        duration: int = 5,
        aspect_ratio: str = "16:9"
    ) -> dict:
        """Génère une vidéo avec Gen-3."""
        response = requests.post(
            f"{self.base_url}/generate/video",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={
                "model": "gen-3",
                "prompt": prompt,
                "duration": duration,
                "aspect_ratio": aspect_ratio,
                "watermark": False
            }
        )

        return response.json()

    def image_to_video(
        self,
        image_url: str,
        motion_prompt: str,
        duration: int = 5
    ) -> dict:
        """Anime une image en vidéo."""
        response = requests.post(
            f"{self.base_url}/generate/image-to-video",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={
                "model": "gen-3",
                "image_url": image_url,
                "prompt": motion_prompt,
                "duration": duration
            }
        )

        return response.json()

    def extend_video(
        self,
        video_url: str,
        extension_prompt: str,
        direction: str = "forward"  # ou "backward"
    ) -> dict:
        """Étend une vidéo existante."""
        response = requests.post(
            f"{self.base_url}/generate/extend",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={
                "video_url": video_url,
                "prompt": extension_prompt,
                "direction": direction
            }
        )

        return response.json()

# Utilisation
runway = RunwayAPI("your-api-key")

# Text-to-video
result = runway.generate_video(
    "A serene lake at sunrise with mist rising from the water"
)

# Image-to-video
result = runway.image_to_video(
    image_url="https://example.com/landscape.jpg",
    motion_prompt="Camera slowly pans right, birds fly across the sky"
)

import torch
from diffusers import StableVideoDiffusionPipeline
from PIL import Image

# Charger le modèle
pipe = StableVideoDiffusionPipeline.from_pretrained(
    "stabilityai/stable-video-diffusion-img2vid-xt",
    torch_dtype=torch.float16,
    variant="fp16"
)
pipe = pipe.to("cuda")

def image_to_video(
    image_path: str,
    num_frames: int = 25,
    fps: int = 7
) -> list:
    """Anime une image en vidéo."""
    # Charger et redimensionner l'image
    image = Image.open(image_path)
    image = image.resize((1024, 576))

    # Générer les frames
    generator = torch.Generator("cuda").manual_seed(42)

    frames = pipe(
        image,
        num_frames=num_frames,
        num_inference_steps=25,
        decode_chunk_size=8,
        generator=generator
    ).frames[0]

    return frames

def save_video(frames: list, output_path: str, fps: int = 7):
    """Sauvegarde les frames en vidéo MP4."""
    from diffusers.utils import export_to_video
    export_to_video(frames, output_path, fps=fps)

# Exemple
frames = image_to_video("landscape.jpg", num_frames=25)
save_video(frames, "animated_landscape.mp4", fps=7)

ARCHITECTURE SORA (DiT) :

1. ENCODAGE
   ┌─────────────────────────────────────────────────┐
   │  Vidéo → Patches spatio-temporels → Latents    │
   │                                                  │
   │  [Frame 1] [Frame 2] ... [Frame N]              │
   │      ↓         ↓            ↓                   │
   │  [Patches 3D divisés en tokens]                 │
   │      ↓                                          │
   │  [VAE Encoder → Espace latent compressé]        │
   └─────────────────────────────────────────────────┘

2. DIFFUSION TRANSFORMER
   ┌─────────────────────────────────────────────────┐
   │  Latents bruitées + Texte encodé (T5/CLIP)     │
   │      ↓                                          │
   │  [Transformer Blocks avec attention 3D]         │
   │  - Self-attention spatiale                      │
   │  - Self-attention temporelle                    │
   │  - Cross-attention texte                        │
   │      ↓                                          │
   │  [Prédiction du bruit]                          │
   └─────────────────────────────────────────────────┘

3. DÉCODAGE
   ┌─────────────────────────────────────────────────┐
   │  Latents débruitées                             │
   │      ↓                                          │
   │  [VAE Decoder → Pixels haute résolution]        │
   │      ↓                                          │
   │  Vidéo finale                                   │
   └─────────────────────────────────────────────────┘

COMPRESSION VIDÉO POUR IA :

VIDÉO ORIGINALE (Trop lourde pour le Transformer)
│
│  1080p × 60 frames × 3 channels = 373 million pixels!
│
├──→ COMPRESSION SPATIALE (VAE)
│    │
│    │  1080p → 128×72 latent (8× réduction par dimension)
│    │
│    └──→ COMPRESSION TEMPORELLE
         │
         │  60 frames → 15 keyframes (4× réduction)
         │
         └──→ TOKENS GÉRABLES
              │
              │  128 × 72 × 15 × 4 channels = 552,960 tokens
              │  (vs 373M pixels originaux)

# Exemple conceptuel de contrôle avancé

def generate_controlled_video(
    prompt: str,
    reference_video: str = None,
    style_image: str = None,
    camera_motion: str = None,
    subject_motion: str = None
) -> str:
    """Génère une vidéo avec contrôles fins."""

    controls = {
        "prompt": prompt,
        "negative_prompt": "blurry, low quality, distorted",
    }

    # Contrôle par vidéo de référence
    if reference_video:
        controls["motion_reference"] = reference_video
        controls["motion_strength"] = 0.7

    # Contrôle de style par image
    if style_image:
        controls["style_reference"] = style_image
        controls["style_strength"] = 0.5

    # Contrôle de caméra
    if camera_motion:
        # "pan_left", "pan_right", "zoom_in", "zoom_out",
        # "orbit", "dolly", "static"
        controls["camera"] = {
            "motion": camera_motion,
            "speed": 1.0
        }

    # Contrôle du sujet
    if subject_motion:
        controls["subject"] = {
            "motion": subject_motion,  # Description du mouvement
            "consistency": 0.9  # Cohérence du sujet
        }

    return api.generate(**controls)

# Exemples d'utilisation
video = generate_controlled_video(
    prompt="A woman in a red dress",
    camera_motion="orbit",
    subject_motion="slowly turns to face camera, smiles"
)

video = generate_controlled_video(
    prompt="Futuristic city street",
    style_image="blade_runner_still.jpg",
    camera_motion="dolly_forward"
)

def video_style_transfer(
    input_video: str,
    style_prompt: str,
    strength: float = 0.7
) -> str:
    """Applique un style à une vidéo existante."""

    # Charger et analyser la vidéo source
    video_analysis = analyze_video(input_video)

    # Générer avec guidance de la vidéo originale
    result = api.generate(
        prompt=style_prompt,
        init_video=input_video,
        denoising_strength=strength,  # 0 = identique, 1 = nouveau
        preserve_motion=True,
        preserve_composition=True
    )

    return result

# Exemple : transformer une vidéo en animation
result = video_style_transfer(
    "real_footage.mp4",
    "Studio Ghibli anime style, hand-drawn, pastel colors",
    strength=0.75
)

from diffusers import StableVideoDiffusionPipeline
import torch

class VideoEnhancer:
    def __init__(self):
        # Modèle d'interpolation
        self.interpolation_model = self._load_interpolation_model()
        # Modèle de super-résolution
        self.sr_model = self._load_sr_model()

    def interpolate_frames(
        self,
        frames: list,
        multiplier: int = 2
    ) -> list:
        """Double (ou plus) le framerate."""
        interpolated = []

        for i in range(len(frames) - 1):
            interpolated.append(frames[i])

            # Générer frames intermédiaires
            for j in range(1, multiplier):
                t = j / multiplier
                intermediate = self.interpolation_model(
                    frames[i], frames[i+1], t
                )
                interpolated.append(intermediate)

        interpolated.append(frames[-1])
        return interpolated

    def upscale_video(
        self,
        frames: list,
        scale: int = 2
    ) -> list:
        """Augmente la résolution de la vidéo."""
        upscaled = []

        for frame in frames:
            high_res = self.sr_model(frame, scale=scale)
            upscaled.append(high_res)

        return upscaled

    def enhance_video(
        self,
        input_path: str,
        output_path: str,
        target_fps: int = 60,
        target_resolution: tuple = (1920, 1080)
    ):
        """Pipeline complet d'amélioration vidéo."""
        frames = self.load_video(input_path)
        original_fps = self.get_fps(input_path)

        # Interpolation pour augmenter le FPS
        fps_multiplier = target_fps // original_fps
        if fps_multiplier > 1:
            frames = self.interpolate_frames(frames, fps_multiplier)

        # Super-résolution
        current_res = frames[0].size
        if current_res[0] < target_resolution[0]:
            scale = target_resolution[0] // current_res[0]
            frames = self.upscale_video(frames, scale)

        self.save_video(frames, output_path, target_fps)

def create_social_media_video(
    concept: str,
    platform: str = "tiktok",
    duration: int = 15
) -> str:
    """Crée une vidéo adaptée aux réseaux sociaux."""

    # Formats par plateforme
    formats = {
        "tiktok": {"ratio": "9:16", "duration": 15, "style": "dynamic, trendy"},
        "instagram_reel": {"ratio": "9:16", "duration": 30, "style": "aesthetic"},
        "youtube_short": {"ratio": "9:16", "duration": 60, "style": "engaging"},
        "twitter": {"ratio": "16:9", "duration": 30, "style": "informative"}
    }

    config = formats.get(platform, formats["tiktok"])

    prompt = f"""
    {concept}

    Style: {config['style']}, social media ready, eye-catching,
    perfect for {platform}, trending aesthetic
    """

    video = api.generate(
        prompt=prompt,
        duration=min(duration, config["duration"]),
        aspect_ratio=config["ratio"]
    )

    return video

# Exemples
create_social_media_video(
    "A coffee being poured in slow motion with steam rising",
    platform="instagram_reel"
)

create_social_media_video(
    "A tech product reveal with dramatic lighting",
    platform="youtube_short"
)

def create_product_video(
    product_image: str,
    product_name: str,
    features: list[str],
    mood: str = "premium"
) -> list[str]:
    """Crée plusieurs clips vidéo pour un produit."""

    moods = {
        "premium": "luxury, elegant, soft lighting, minimalist",
        "energetic": "dynamic, vibrant colors, fast cuts",
        "tech": "futuristic, neon accents, sleek, innovative",
        "natural": "organic, earthy tones, soft focus, authentic"
    }

    style = moods.get(mood, moods["premium"])

    videos = []

    # Hero shot
    videos.append(api.image_to_video(
        image=product_image,
        prompt=f"{product_name} rotating slowly, {style}, product showcase",
        duration=5
    ))

    # Feature highlights
    for feature in features[:3]:
        videos.append(api.generate(
            prompt=f"{product_name} demonstrating {feature}, {style}",
            duration=3
        ))

    # Call-to-action
    videos.append(api.generate(
        prompt=f"{product_name} with lifestyle context, {style}, aspirational",
        duration=5
    ))

    return videos

def create_storyboard_video(
    storyboard: list[dict],  # [{"image": ..., "description": ..., "duration": ...}, ...]
    style: str = "cinematic"
) -> str:
    """Transforme un storyboard en pré-visualisation vidéo."""

    clips = []

    for i, panel in enumerate(storyboard):
        # Générer le clip pour ce panel
        if panel.get("image"):
            # Animer l'image du storyboard
            clip = api.image_to_video(
                image=panel["image"],
                prompt=f"{panel['description']}, {style}",
                duration=panel.get("duration", 3)
            )
        else:
            # Générer à partir de la description
            clip = api.generate(
                prompt=f"{panel['description']}, {style}",
                duration=panel.get("duration", 3)
            )

        clips.append(clip)

    # Concaténer les clips
    final_video = concatenate_videos(clips)

    return final_video

# Exemple de storyboard
storyboard = [
    {
        "description": "Wide shot of a dark forest at night, moon visible",
        "duration": 4
    },
    {
        "description": "A mysterious figure emerges from the shadows",
        "duration": 3
    },
    {
        "image": "character_concept.jpg",
        "description": "Close-up of the character's face, determined expression",
        "duration": 2
    }
]

preview = create_storyboard_video(storyboard, style="dark fantasy, cinematic")

ANATOMIE D'UN PROMPT VIDÉO :

1. SUJET PRINCIPAL
   "A golden retriever"

2. ACTION / MOUVEMENT
   "running through a field of flowers"

3. ENVIRONNEMENT
   "in a sunny meadow, mountains in background"

4. ÉCLAIRAGE
   "golden hour lighting, lens flare"

5. MOUVEMENT DE CAMÉRA
   "tracking shot following the dog"

6. STYLE VISUEL
   "cinematic, 35mm film, shallow depth of field"

7. QUALITÉ
   "high quality, 4K, smooth motion"

PROMPT COMPLET :
"Tracking shot following a golden retriever running joyfully
through a field of colorful wildflowers. Sunny meadow with
snow-capped mountains in the distant background. Golden hour
lighting with natural lens flare. Cinematic 35mm film look,
shallow depth of field, smooth slow motion. High quality 4K."

CAMERA_MOTIONS = {
    "static": "static camera, locked off shot",
    "pan_left": "camera slowly pans from right to left",
    "pan_right": "camera slowly pans from left to right",
    "tilt_up": "camera tilts upward",
    "tilt_down": "camera tilts downward",
    "zoom_in": "slow zoom in, getting closer",
    "zoom_out": "slow zoom out, revealing more of the scene",
    "dolly_in": "camera moves forward through the scene",
    "dolly_out": "camera pulls back through the scene",
    "orbit": "camera orbits around the subject",
    "crane_up": "camera rises up like a crane shot",
    "crane_down": "camera descends from above",
    "tracking": "camera follows the subject's movement",
    "handheld": "slight handheld camera movement, organic feel",
    "drone": "aerial drone shot, bird's eye view"
}

def build_video_prompt(
    subject: str,
    action: str,
    environment: str,
    camera: str = "static",
    style: str = "cinematic"
) -> str:
    camera_desc = CAMERA_MOTIONS.get(camera, camera)

    return f"""
    {subject} {action} in {environment}.
    {camera_desc}.
    {style} style, high quality, smooth motion, professional cinematography.
    """

DÉFIS DE LA GÉNÉRATION VIDÉO :

1. COHÉRENCE TEMPORELLE
   ├── Objets qui changent entre frames
   ├── Physique incohérente
   └── Mouvements non naturels

2. MAINS ET VISAGES
   ├── Doigts incorrects
   ├── Expressions faciales étranges
   └── Mouvements de lèvres décalés

3. TEXTE DANS LES VIDÉOS
   ├── Lettres illisibles
   ├── Texte qui change
   └── Signalétique incohérente

4. INTERACTIONS PHYSIQUES
   ├── Objets qui se traversent
   ├── Collisions incorrectes
   └── Reflets/ombres incohérents

5. DURÉE ET COÛT
   ├── Génération lente (minutes par seconde)
   ├── Coûts élevés par vidéo
   └── Résolution encore limitée

# 1. Être spécifique sur le mouvement
# MAUVAIS : "A bird flying"
# BON : "A single white dove gliding smoothly from left to right,
#        wings fully extended, against a clear blue sky"

# 2. Limiter la complexité
# MAUVAIS : "A busy city street with hundreds of people,
#           cars, and a parade passing by"
# BON : "A quiet city street at dawn, a single cyclist
#        rides past the camera"

# 3. Spécifier la caméra
# MAUVAIS : "A waterfall"
# BON : "A majestic waterfall, static wide shot,
#        mist rising, camera locked off"

# 4. Contrôler le rythme
# MAUVAIS : "An action sequence"
# BON : "Slow motion shot of water droplets splashing,
#        each droplet clearly visible, time seems frozen"

# 5. Éviter les visages proches
# MAUVAIS : "Extreme close-up of a person talking"
# BON : "Medium shot of a silhouette figure walking away,
#        back to camera, mysterious atmosphere"

GÉNÉRATION VIDÉO IA :

MODÈLES PRINCIPAUX
├── Sora (OpenAI) : Référence, 60s+, 1080p
├── Runway Gen-3 : Web/API, versatile
├── Stable Video : Open source
└── Veo/Kling/Pika : Alternatives

TECHNIQUES
├── Text-to-video : Génération depuis texte
├── Image-to-video : Animation d'images
├── Video-to-video : Style transfer
└── Extension : Prolonger des vidéos

ARCHITECTURE
├── Diffusion Transformers (DiT)
├── Compression spatio-temporelle
├── Attention 3D
└── VAE pour encodage/décodage

CAS D'USAGE
├── Réseaux sociaux
├── Publicité/Marketing
├── Prototypage film
├── Éducation/Formation
└── Art et créativité

LIMITES
├── Cohérence temporelle
├── Mains/visages
├── Physique réaliste
└── Coût et vitesse