IJCAI 2017 paper presentation

1. Tensor
Decomposi-on
with
Missing
Indices
Yuto
Yamaguchi
and
Kohei
Hayashi
17/08/22
IJCAI2017@Melbourne
1

2. Tensor
data
17/08/22
IJCAI2017@Melbourne
2
#
#
(userA,
#movie,
Melbourne):
1
(userB,
#tennis,
Sydney):
2
(userC,
#dinner,
Canberra):
1
(userB,
#beer,
Brisbane):
1
(userA,
#dinner,
Melbourne):
2
e.g.,
TwiNer
data
(user,
hashtag,
loca-on)
Tensor
data
=
mul--­‐dimensional
data
value

3. Tensor
decomposi-on
17/08/22
IJCAI2017@Melbourne
3
e.g.,
CP
decomposi-on
[Carroll
and
Chang,
1970]
+
+
…
=
Applica-ons
• Recommenda-ons,
noise
reduc-on,
data
compression,
…
ˆXijk = UirVjrWkr
r
∑
X
V:,
1
U:,
1
W:,
1
V:,
2
U:,
2
W:,
2

4. [Our
problem]
what
if
indices
are
missing?
17/08/22
IJCAI2017@Melbourne
4
#
#
(userA,
#movie,
Melbourne):
1
(userB,
#tennis,
Sydney):
2
(userC,
#dinner,
Canberra):
1
(userB,
#beer,
-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐):
1
(userA,
-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐,
Melbourne):
2
Conven5onal
tensor
decomposi5on
algorithms
do
not
apply
to
these
“incomplete
samples”
L
value

5. [Our
problem]
what
if
indices
are
missing?
17/08/22
IJCAI2017@Melbourne
5
#
#
(userA,
#movie,
Melbourne):
1
(userB,
#tennis,
Sydney):
2
(userC,
#dinner,
Canberra):
1
(userB,
#beer,
-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐):
1
(userA,
-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐,
Melbourne):
2
Conven5onal
tensor
decomposi5on
algorithms
do
not
apply
to
these
“incomplete
samples”
L
value
Values
are
not
missing

6. PROPOSED
MODEL
17/08/22
IJCAI2017@Melbourne
6

7. Basic
idea
17/08/22
IJCAI2017@Melbourne
7
(userA,
#movie,
Melbourne)
(userB,
#tennis,
Sydney)
(userC,
#dinner,
Canberra)
(userB,
#beer,
Brisbane)
(userA,
#dinner,
Melbourne)
+
+
…
e.g.,
CPD
infer
construct
decompose
Solve
tensor
decomposi5on
and
missing
indices
inference
repeatedly

8. Proposed
model
(1/2)
17/08/22
IJCAI2017@Melbourne
8
Handle
indices
as
unobserved
variables
ˆin ∈ 1,2,…I,φ{ }
Observed
(can
be
missing)
indices
True
(unobserved)
indices
missing
Tensor
elements
Decomposi5on
parameters
[3rd-­‐order
case]

9. Proposed
model
(2/2)
17/08/22
IJCAI2017@Melbourne
9
1.
Generate
decomposi-on
parameters
depending
on
the
decomposi-on
model
Θ = U,V,W{ } Uir = N ⋅ 0,
1
λ
"
#
$
%
&
' for
all
i
and
r
e.g.,
CPD

10. Proposed
model
(2/2)
17/08/22
IJCAI2017@Melbourne
10
2.
Generate
N
indices
(in,
jn,
kn)
Delta
if
not
missing
Uniform
if
missing
in ~

11. Proposed
model
(2/2)
17/08/22
IJCAI2017@Melbourne
11
3.
Generate
N
tensor
elements
depending
on
decomposi-on
model
e.g.,
CPD
ˆXin jnkn
= UinrVjnrWknr
r
∑

12. Proposed
model
is
a
natural
extension
of
the
conven-onal
tensor
decomposi-on
17/08/22
IJCAI2017@Melbourne
12
where
MLE
Θ
of
the
proposed
model

13. Parameter
inference
Varia-onal
MAP-­‐EM
algorithm
• E-­‐step
– Missing
indices
are
inferred
using
learnt
tensor
decomposi-on
• M-­‐step
– Tensor
decomposi-on
is
learnt
using
inferred
indices
17/08/22
IJCAI2017@Melbourne
13
See
the
paper
for
details
if
interested
J

14. Time
Complexity
(Mth-­‐order
tensor)
17/08/22
IJCAI2017@Melbourne
14
Proposed
algorithm
for
CPD
Conven-onal
CPD
N
Nm
-
R
Im
:
#
of
samples
:
#
of
missing
indices
for
mth
mode
:
#
of
latent
dimensions
:
#
of
dimensions
for
mth
mode
Only
addi5onal
term

15. EXPERIMENTS
17/08/22
IJCAI2017@Melbourne
15

16. Compared
algorithms
17/08/22
IJCAI2017@Melbourne
16
[MAP-­‐EM]:
Proposed
algo.
with
q
inferred
[Uniform]:
Proposed
algo.
with
q
fixed
as
uniform
[Prior]:
Proposed
algo.
with
q
fixed
as
data
histogram
[Minimal]:
CPD
with
only
complete
samples
[Complete]:
CPD
with
only
complete
modes
[CMTF]:
Coupled
matrix
tensor
factoriza-on
[Acar+,
2011]
Approx.
distribu5on
on
varia5onal
inference
Proposed
Baselines

17. Results
17/08/22
IJCAI2017@Melbourne
17
Lower
beNer
Lower
beNer
Upper
beNer
Proposed
model
(red)
works
well
if
• the
number
of
samples
is
large,
or
• missing
ra-o
is
not
very
large
Synthe5c
data
generated
by
our
model
TwiZer
data
(user,
hashtag,
loca5on)
sample
size
large
(n=10)
sample
size
small
(n=1)

18. Summary
• [New problem]
– Defined a new tensor decomposition problem where
the indices are partially missing
• [Model]
– Proposed a probabilistic generative model to handle
missing indices
• [Algorithm]
– Developed a parameter inference algorithm
17/08/22
IJCAI2017@Melbourne
18
Github: yamaguchiyuto/missing_tensor_decomposition