In a listing, how to have line numbers not in margin?

To shift the numbers but not the caption requires to dig deeper into the listings package.

Instead of the \lst@PlaceNumber command without arguments we define \lst@FormatNumber which takes the number as an argument. This allows us to print also nothing but occupying the same space. We define a new value for the option numbers called LEFT for numbers not in the margin; most of the code is a copy of the original code.

\lst@Key{numbers}{none}{%
    \def\lst@FormatNumber##1{\relax}%   <<<<<<<<<<<<<<<<<<
    \lstKV@SwitchCases{#1}%
    {none&\\%
     left&\def\lst@FormatNumber##1{\llap{\normalfont % <<<<<<<<<<<<<
                \lst@numberstyle{##1}\kern\lst@numbersep}}\\%
     right&\def\lst@FormatNumber##1{\rlap{\normalfont % <<<<<<<<<<<<<
                \kern\linewidth \kern\lst@numbersep
                \lst@numberstyle{##1}}}\\%
     LEFT&\def\lst@FormatNumber##1{\makebox[\lst@LEFTmargin][r]{\normalfont  % <<<<<<<<<<<<<<<<<<<<<<<<<<
                \lst@numberstyle{##1}\kern\lst@numbersep}}% <<<<<<<<<<
    }{\PackageError{Listings}{Numbers #1 unknown}\@ehc}}

Moreover, we have to extend the printing of the number by an else branch such that an 'empty' number is printed in unnumbered lines.

\gdef\lst@SkipOrPrintLabel{%
    \ifnum\lst@skipnumbers=\z@
        \global\advance\lst@skipnumbers-\lst@stepnumber\relax
        \lst@FormatNumber{\thelstnumber}%     <<<<<<<<<<<<<<<<<
        \lst@numberfirstlinefalse
    \else
        \lst@ifnumberfirstline
            \lst@FormatNumber{\thelstnumber}% <<<<<<<<<<<<<<<<<
            \lst@numberfirstlinefalse
        \else                               % <<<<<<<<<<<<<<<<<
            \lst@FormatNumber{}%              <<<<<<<<<<<<<<<<<
        \fi
    \fi
    \global\advance\lst@skipnumbers\@ne
}

Finally, to control the width reserved for the number, we introduce a new key LEFTmargin with default value 2em.

\lst@Key{LEFTmargin}{2em}{\def\lst@LEFTmargin{#1}}

If these definitions are included in the preamble instead of in a file loaded by \usepackage, they have to be enclosed by \makeatletter and \makeatother.

Here is the sample code from the original posting.

\documentclass[english,conference]{IEEEtran}
\usepackage{lmodern}
\usepackage{listings}
\lstset{captionpos=b}
\lstset{basicstyle=\small\ttfamily}
\lstset{showstringspaces=false, columns=flexible, keepspaces=true}
\lstset{tabsize=2, gobble=2}
\lstset{numbers=left, numberstyle=\tiny, numbersep=5pt,
   numberfirstline=true, firstnumber=1, stepnumber=5}
\usepackage{blindtext}

\makeatletter
\lst@Key{numbers}{none}{%
    \def\lst@FormatNumber##1{\relax}%
    \lstKV@SwitchCases{#1}%
    {none&\\%
     left&\def\lst@FormatNumber##1{\llap{\normalfont
                \lst@numberstyle{##1}\kern\lst@numbersep}}\\%
     right&\def\lst@FormatNumber##1{\rlap{\normalfont
                \kern\linewidth \kern\lst@numbersep
                \lst@numberstyle{##1}}}\\%
     LEFT&\def\lst@FormatNumber##1{\makebox[\lst@LEFTmargin][r]{\normalfont
                \lst@numberstyle{##1}\kern\lst@numbersep}}%
    }{\PackageError{Listings}{Numbers #1 unknown}\@ehc}}
\gdef\lst@SkipOrPrintLabel{%
    \ifnum\lst@skipnumbers=\z@
        \global\advance\lst@skipnumbers-\lst@stepnumber\relax
        \lst@FormatNumber{\thelstnumber}%
        \lst@numberfirstlinefalse
    \else
        \lst@ifnumberfirstline
            \lst@FormatNumber{\thelstnumber}%
            \lst@numberfirstlinefalse
        \else
            \lst@FormatNumber{}%
        \fi
    \fi
    \global\advance\lst@skipnumbers\@ne
}
\lst@Key{LEFTmargin}{2em}{\def\lst@LEFTmargin{#1}}
\makeatother

\begin{document}
\title{The Title}
\author{author}
\maketitle
\begin{abstract}
\blindtext
\end{abstract}
\blindtext[4]
\begin{figure}
    \centering
    mah figure
    \caption{This is a very long caption, it is so long that it spans two lines.}
    \label{myfig}
\end{figure}

\begin{lstlisting}[numbers=LEFT,LEFTmargin=1.2em,numbersep=2pt,float,caption={This is a very long caption, it is so long that it spans two lines.},label=mylisting]
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
  teh c0dez
\end{lstlisting}
\end{document}