literature Tree
论文解析树
  Abstract
    Task
      efficiency robustness, latency attack
    Technical Challenge
      no latency attack focusing on large vision-language model(VLMs)
    technical contribution
      contribution1
        three loss function
      contribution2
        tricks
          temporal weight adjustment
    Experiment
      MS-coco & ImageNet
  Introduction
    Task And application
      energy-latency
    previous method
      a larger representation dimension calculation
        maximize the L2 norm
      NICGSlowDown
        increase the number of decoder calls,
        minimize the logits of both EOS token and output tokens
        问题
          targets for RNNs or LSTMs，不是VLM
          现在的VLM 同一个输入 输出也不一样
            challenging to optimize objectives with specific output tokens
    pipeline that fix
      contribution 1
        energy consumption, latency time, and the length of generated sequences
          positively linearly correlated
      contribution2
        craft an imperceptible perturbation to induce high energylatency cost for VLMs
        3 loss function
      contributions
        效果不错
  Method
    Overview
      具体任务：输入输出
方法：第一步第二步第三步
        craft an imperceptible image and induce VLMs to generate a sequence as long as possible,
    3 loss function
      delay EOS
        minimize the probability of the EOS token at all positions
      output uncertainty
          最大化输出分布 等于 最小化KL散度
      token diversity
        increase rank 可以增加diversity
          NP-hard
        为了求解Nuclear Norm
          the nuclear norm of the concatenated matrix of hidden states among all generated tokens
    optimization tricks
      PGD
        projected gradient descent
      temporal weight adjustment algorithm
      momentum
  Experiments
    这篇投的ICLR 实验很多
    set
      MS-COCO\ImageNet
      BLIP
        BLIP-2
          InstructBLIP
            MIniGPT-4
              using Lavis
    metrics
      energy consumtion
      latency time
      response time cost
      length of generated sequences
      NVML
    Comparison experiments
    其他实验
      幻觉率
        CHAIR
      GradCAM
        观察注意力分布变化
      blackbox attack
        分别把一个当作surrogate model
      其他任务
        VQA、Visual Reason
      BLEU值
      embedding distance
      different sampling policies
      different maximum lengths.
    Ablation studies
      loss function
        三个loss 加起来最好
      decay+momentum
      perturbation magnitudes
    grid search