Vim, 27, 26 bytes

DJqq:t$|s/.\zs[^<C-r>"]<CR>@qq@qD

Try it online!

Input comes in this format:

e
codegolf.stackexchange.comk

I'm very proud of this answer because I don't typically use ex commands in my vim answers. My naive first approach is three bytes longer:

i:s/.\zs[^<Right>]<Esc>"addqqYp@a@qq@qdd

Less successful approaches:

i:s/.\zs[^<Right>]<Esc>"addqqYp@a@qq@qdd
i:s/.\zs[^<Right>]<CR>@q<Esc>"adkqqYp@aq@qdd
DJ:s/.\zs[^<C-r>"]<CR>uqqYp@:@qq@qdd
DJqq:t.|$s/.\zs[^<C-r>"]<CR>@qq@qdd

Explanation:

D                                   " Delete everything on this first line
 J                                  " Remove this empty line
  qq                                " Start recording into register 'q'
    :                               " Open the vim command line (for 'ex' commands)
     t$                             " Make another copy of this line
       |                            " And
         s/                         " Run a substitute command on the last line in the buffer. Remove:
           .                        "   Any character
            \zs                     "   Move the selection forward (so we don't delete the first one)
               [^<C-r>"]            "   Followed by anthing *except* for the character we deleted
                        <CR>        " Run the command
                            @q      " Call register 'q'
                              q     " Stop recording
                               @q   " Run the recursive macro
                                 D  " Delete our extra line

Haskell, 50 bytes

w@(a:x)%c|(d,_:y)<-span(==c)x=w:(a:d++y)%c|0<1=[w]

Retina, 28 27 bytes

Byte count assumes ISO 8859-1 encoding.

;{G*1`
R1r`(?!^|.*¶?\1$)(.)

Try it online!

Explanation

;{G*1`

There's a lot of configuration here. The stage itself is really just G1`, which keeps only the first line, discarding the input character. * turns it into a dry run, which means that the result (i.e. the first line of the string) is printed without actually changing the string. { tells Retina to run both stages in a loop until the string stops changing and ; prevents output at the end of the program.

R1r`(?!^|.*¶?\1$)(.)

This discards the first character which a) isn't at the beginning of the input, b) isn't equal to the separate input character.

MATL, 20 16 bytes

y-f1X-"t[]@X@q-(

Try it online! Or verify test cases: 1, 2, 3, 4, 5.

Bonus: modified code to see the string being gradually shrunk (offline compiler):

y-f1X-"tt.3Y.XxD[]@X@q-(].3Y.XxDvx

Explanation

y        % Implicitly input string and char. Duplicate string onto top
-        % Subtract. Gives nonzero for chars in the input string that are
         % different from the input char
f        % Array of indices of nonzero values
1X-      % Remove 1 from that array. This gives an array of the indices of 
         % chars to be removed from the string
"        % For each
  t      %   Duplicate previous string
  []     %   Push empty array
  @      %   Push index of char to be removed. But this index needs to be 
         %   corrected to account for the fact that previous chars have
         %   already been removed...
  X@q-   %   ... So we correct by subtracting the 0-based iteration index
  (      %   Assign empty array to that position, to remove that char
         % Implicitly end for each
         % Implicitly display

Perl 5, 29 bytes

I got 35 bytes using Strawberry Perl: 31 bytes, plus 1 for -nE instead of -e, plus 3 for space + -i (used for the single-letter input; the longer string is from STDIN).

chomp;say;s/(.)[^$^I]/$1/&&redo

However, I've no doubt this is doable without chomp; using <<<, which is 29 bytes, even though I can't test it myself using Strawberry.

say;s/(.)[^$^I]/$1/&&redo

Thus:

perl -im -nE'say;s/(.)[^$^I]/$1/&&redo' <<< "example"

Perl 6,  47 40  38 bytes

->\a,\b{a,{S/^(."{b}"*:)./$0/}...^{$^a eq $^b}}

->\a,\b{a,{S/^(."{b}"*:)./$0/}...^&[eq]}

{$^b;$^a,{S/^(."$b"*:)./$0/}...^&[eq]}

Expanded:

{       # lambda with two placeholder parameters ｢$a｣ and ｢$b｣

  $^b;    # declare second parameter

  $^a,    # declare first parameter, and use it to seed the sequence

  {       # bare block lambda with implicit parameter ｢$_｣
    S/      # string replace and return
      ^       # beginning of string
      (       # capture into ｢$0｣
        .       # the first character
        "$b"*   # as many ｢$b｣ as possible
        :       # don't allow backtracking
      )
      .       # any character ( the one to be removed )

    /$0/      # put the captured values back into place
  }

  ...^      # repeat that until: ( and throw away the last value )

  &[eq]     # the infix string equivalence operator/subroutine

}

The reason ...^ was used instead of ... is that &[eq] wouldn't return True until the last value was repeated.

Pip, 22 26 24 22 bytes

Lv+#Paa@oQb?++oPaRA:ox

Takes string as first command-line argument, character as second. Try it online!

Explanation

Loops over characters of input; if the character equals the special character, move on to the next one; if not, delete it and print the string.

An ungolfed version (a, b get cmdline args; o starts with a value of 1, x is ""):

P a         Print a
L #a-1      Loop len(a)-1 times:
 I a@o Q b   If a[o] string-eQuals b:
  ++o         Increment o
 E {         Else:
  a RA: o x   In-place in a, Replace char At index o with x (i.e. delete it)
  P a         Print a
 }

Golfing tricks:

• The loop header for L is only evaluated once, so we can sneak the initial print in there. #Pa-1 won't work because P is low-precedence (it would parse as #P(a-1)), but we can rearrange it to v+#Pa, using the v variable preinitialized to -1.
• The RA: operator returns the new value of a, so we can print that expression instead of having a separate Pa statement.
• Now both of the branches of the if statement are single expressions, so we can use the ternary operator ? instead.

05AB1E, 26 25 bytes

¬ˆ[¦Ðg_#¬²k0Qi²ˆë¯J?,]¯J?

¬ˆ                         Put the first character into an array
  [                        While true
   ¦                       Remove the first character
    Ð                      Triplicate
     g_#                   if the string is empty, break
        ¬²k0Qi             if the first character is equal to the one specified in the input
              ²ˆ           Add it to the array
                ë          Else
                 ¯J?       Display the array
                    ,      Display the remaining string
                     ]     End while
                      ¯J?  Display the last string

Try it online!

Please note that ¬²k0Q could be rewritten ¬²Q, but for some reason it doesn't work when the current character is a quote mark: Q returns the actual string instead of a boolean and it causes an infinite loop.

This code can be golfed further since ¯J? is duplicated. Moving this part in the loop would remove the duplication and would also allow to drop the closing square bracket.

Python 2, 71 66 bytes:

f,m=input();k=f[0]
while f:a=f[0]==m;k+=f[0]*a;f=f[1+a:];print k+f

A full program. Takes 2 inputs through STDIN in the format '<String>','<Char>'.

Also, here is a recursive solution currently at 140 bytes:

Q=lambda c,p,k='',j=1,l=[]:c and Q(c[1:],p,k+c[0]*(j<2)+c[0]*(c[0]==p),j+1,l+[k+c])or'\n'.join(sorted({*l},key=l.index))+('\n'+k)*(k not in l)

This one should be called in the format print(Q('<String>','<Char>')).

JavaScript (ES6), 74 bytes

(s,c)=>[...t=s.slice(1)].map(d=>c!=d?s+=`
`+s[0]+(t=t.replace(d,``)):s)&&s

c90, 129 125 bytes

with whitespace:

main(q, x)
{
    for (char **v = x, *a = v[1], *b = a, *c;*b++;)
        for (c = a; c == a | *c == *v[2] && *b != *v[2] && putchar(*c),
            ++c != b || *b != *v[2] && !puts(b););
}

without whitespace:

main(q,x){for(char**v=x,*a=v[1],*b=a,*c;*b++;)for(c=a;c==a|*c==*v[2]&&*b!=*v[2]&&putchar(*c),++c!=b||*b!=*v[2]&&!puts(b););}

ungolfed:

#include <stdio.h>
int main(int argc, char **argv)
{
    char *a = argv[1];
    for (char *b = a + 1; *b; b++) {
        if (*b == *argv[2]) {
            continue;
        }
        putchar(*a);
        for (char *c = a + 1; c != b; c++) {
            if (*c == *argv[2]) {
                putchar(*c);
            }
        }
        puts(b);
    }
}

This takes a pointer to the start of the string, and loops, iterating this pointer until it reaches the end of the string. Within the loop, it prints the first character, then any instances of the second argument it finds between the start of the string and the pointer. After this, it calls puts on the pointer, printing out the rest of the string.

This must be compiled on a system where sizeof(int) == sizeof(char*). +3 bytes otherwise.

This is the first time I've tried code golfing here, so I'm sure there are some optimizations to be made.

Java 7, 155 140 bytes

void c(String s,char c){System.out.println(s);for(int i=1;i<s.length();)if(s.charAt(i++)!=c){c(s.substring(0,i-1)+s.substring(i),c);break;}}

Ungolfed:

void c(String s, char c){
  System.out.println(s);
  for(int i = 1; i < s.length();){
    if(s.charAt(i++) != c){
      c(s.substring(0, i-1) + s.substring(i), c);
      break;
    }
  }
}

Test code:

Try it here.

class M{
  static void c(String s,char c){System.out.println(s);for(int i=1;i<s.length();)if(s.charAt(i++)!=c){c(s.substring(0,i-1)+s.substring(i),c);break;}}

  public static void main(String[] a){
    c("Make a \"Ceeeeeeee\" program", 'e');
  }
}

Output:

Make a "Ceeeeeeee" program
Mke a "Ceeeeeeee" program
Me a "Ceeeeeeee" program
Mea "Ceeeeeeee" program
Me "Ceeeeeeee" program
Me"Ceeeeeeee" program
MeCeeeeeeee" program
Meeeeeeeee" program
Meeeeeeeee program
Meeeeeeeeeprogram
Meeeeeeeeerogram
Meeeeeeeeeogram
Meeeeeeeeegram
Meeeeeeeeeram
Meeeeeeeeeam
Meeeeeeeeem
Meeeeeeeee

Dyalog APL, 27 bytes

{×i←⊃1+⍸⍺≠1↓⎕←⍵:⍺∇⍵/⍨i≠⍳≢⍵}

⍺ is the excluded character, ⍵ is the initial string

print argument; find index i of the first non-⍺ after the first char; if found, call recursively with i removed

C#, 122 117 112 bytes

IEnumerable F(string s,char c){for(int i=0;i<s.Length;++i)if(i<1||s[i]!=c)yield return i>0?s=s.Remove(i--,1):s;}

Ungolfed :

public IEnumerable F(string s, char c) {
    for (int i = 0; i < s.Length; ++i) {
        if (i < 1 || s[i] != c)
            yield return i > 0 ? s = s.Remove(i--, 1) : s;
    }
}

Returns a collection of strings.

Pyke, 26 19 bytes

jlFjR<Q/Q*jhR+jih>+

Try it here!

05AB1E, 26 24 23 bytes

Thanks @Kade for 2 bytes!
Thanks @Emigna for 1 byte!

¬UDvy²k0Êiy¡¬s¦yý«Xs«=¦

Uses the CP-1252 encoding. Try it online!

y²k0Ê could be y²Ê but the " messes it up.

This probably could be golfed more because « is repeated twice. Please leave a comment if you have any suggestions or ways to golf it down more.

Ruby, 148 139 97 90 83 77 bytes

a,c=$*;s=a.dup;i=1;p s;while s[i]do if s[i]!=c;s[i]="";p s;else i+=1;end end

Not sure if amateur code is accepted on this exchange but I'm interested in learning to code golf although I'm terrible at it, any help on how I'd get this program looking as small as the others here?

EDIT:

Replaced puts with p

Removed a tonne of whitespace and counted bytes correctly thanks to Wheat Wizard

Thanks to challenger5 went from s=gets.chop;c=gets.chop; to s,c=gets.chop,gets.chop;

replaced then with ; and gets.chop with gets[0] thanks Mhutter!

Taking input as command line variables now, eg. prog.rb helloworld l

I think this is as small as I can manage!

C#, 135 138 :( 137 bytes

Golfed:

IEnumerable<string>F(string s,char c){int i=1,l;for(;;){yield return s;l=s.Length;while(i<l&&s[i]==c)i++;if(i==l)break;s=s.Remove(i,1);}}

Ungolfed:

    IEnumerable<string> F(string s, char c)
    {
        int i = 1, l;

        for (;;)
        {
            yield return s;

            l = s.Length;

            while (i < l && s[i] == c)
                i++;

            if (i == l)
                break;

            s = s.Remove(i, 1);
        }
    }

Function returns collection of strings.

EDIT1: @psycho noticed that algorithm was not implemented properly.

EDIT2: Created variable for s.Length. One byte saved thanks to @TheLethalCoder.

Mathematica, 78 bytes

Damn you Martin Ender, I was almost first :p

(i=2;a={c=#};While[i<=Length@c,If[c[[i]]==#2,i++,c=c~Drop~{i};a=a~Append~c]];a)&

Unnamed function; straightforward implementation with a While loop and a few temporary variables.

JavaScript ES6, 89 bytes

I thought this would be an easy challenge, but I'm pretty sure I'm missing something here..

Uses recursion and returns an array of strings

(c,i=1,r)=>f=a=>a[i]?a[i++]==c?f(a):f(g=a.slice(0,i-1)+a.slice(i--),(r=r||[a]).push(g)):r

F=
  (c,i=1,r)=>f=a=>a[i]?a[i++]==c?f(a):f(g=a.slice(0,i-1)+a.slice(i--),(r=r||[a]).push(g)):r

G=_=>A.value.length>1 && (O.innerHTML = F(E.value)(A.value).join`
`)

G()

<input id=A oninput='G()' value='alphabet'>
<input id=E oninput='G()' value='a'>
<pre id=O>

JavaScript (ES6), 69

Returning an array of strings

s=>c=>[...s].map((x,i,z)=>i?x!=c&&(z[i]='',z.join``):s).filter(x=>x)

Returning a single string with newlines

s=>c=>[...s].map((x,i,z)=>i?x!=c?(z[i]='',`
`+z.join``):'':s).join``

F=
s=>c=>[...s].map((x,i,z)=>i?x!=c?(z[i]='',`
`+z.join``):'':s).join``

function update() {
  var s=S.value,c=C.value[0]
  O.textContent=F(s)(c)
}

update()

<input id=S value='Hello world!' oninput='update()'>
<input id=C value='!' oninput='update()'>
<pre id=O></pre>
                  

Mathematica, 64 bytes

Most@FixedPointList[StringReplace[#,b_~~Except@a:>b,1]&,a=#2;#]&

Anonymous function. Takes two strings as input, and returns a list of strings as output. Works by repeatedly removing the first non-instance of the character.

Python 3, 67 63 Bytes

a,b=input(),input()
print(''.join([a[0]]+[x for x in a if x==b]))

It takes a as the string and b as the char, then goes through each letter in the string and adds it to the list if it's the first one or the character. Then it joins and prints.

EDIT: Fixed the problem mentioned by @R. Kap where it included copies of the first character as well, and shortened it to 63 bytes in the meantime.

PHP, 88 84 86 85 82 81 78 bytes

1 byte saved thanks to @IsmaelMiguel, 3 bytes thanks to @user59178, 3 bytes inspired by @user59178

while($b=substr("$argv[1]\n",$i++))$a>""&$b[0]!=$argv[2]?print$a.$b:$a.=$b[0];

takes input from command line arguments; run with php -r <code> '<string>' <character>

• appends a newline to the input for an implicit final print.
  That adds 5 4 bytes of code, but saves on the output and an additional echo$a;.

C,86 83 bytes

char x[];i;m(char*v,int c){for(x[i]=*v;*v;)*++v-c?printf("%s%s\n",x,v):(x[++i]=c);}

TSQL, 127 bytes(Excluding variable definitions)

DECLARE @1 VARCHAR(100)='codegolf.stackexchange.com'
DECLARE @2 CHAR(1) = 'o'

DECLARE @ char=LEFT(@1,1)WHILE patindex('%[^'+@2+']%',@1)>0BEGIN SET @1=STUFF(@1,patindex('%[^'+@2+']%',@1),1,'')PRINT @+@1 END

Formatted:

DECLARE @1 VARCHAR(100) = 'codegolf.stackexchange.com'
DECLARE @2 CHAR(1) = 'o'
DECLARE @ CHAR = LEFT(@1, 1)

WHILE patindex('%[^' + @2 + ']%', @1) > 0
BEGIN
    SET @1 = STUFF(@1, patindex('%[^' + @2 + ']%', @1), 1, '')

    PRINT @ + @1
END

JavaScript [101 bytes]

f = phrase, t = key

a=[];a.push(f[0]);for(var i=0;i<f.length;i++)if(f[i]==t)a.push(f[i]);for(var p in a)console.log(a[p])

Python 3, 87 83 bytes

s=open(0).read()
p,c,s=s[0],s[-2],s[:-3]
for h in s:p+=s[:h==c];s=s[1:];print(p+s)

Explanation: I chose python 3 instead of python 2, because it allows to use file descriptors in open (and 0 is the descriptor of the standard input). In each loop iteration p keeps track of stored prefix. s[:h==c] works as follows: boolean expression h==c is casted into integer (0 or 1), and then the corresponding amount of characters from the beginning of s are taken.

P.S. The loop in the initial version was:

for h in s:
 if h==c:p+=h
 s=s[1:];print(p+s)

JavaScript (ES2015) - 86 bytes

let f=([a,...s],c,r=[])=>r[0]==a+s?r:f(a+s.replace(eval(`/[^${c}]/`),''),c,[a+s,...r])

Javascript ES6 - 61 Bytes

f=(a,b)=>a[0]+a.substring(1).split(RegExp(`[^${b}]`)).join``;

Accessing strings as arrays saves the day! (And a ton of bytes).