1. Conjuntive Formulation of the Random Set Framework for Multiple Instance Learning:Application to Remote Sensing Jeremy Bolton Paul Gader CSI Laboratory University of Florida

2. Highlights Conjunctive forms of Random Sets for Multiple Instance Learning: Random Sets can be used to solve MIL problem when multiple concepts are present Previously Developed Formulations assume Disjunctive relationship between concepts learned New formulation provides for a conjunctive relationship between concepts and its utility is exhibited on a Ground Penetrating Radar (GPR) data set

3. Outline Multiple Instance Learning MI Problem RSF-MIL Multiple Target Concepts Experimental Results GPR Experiments Future Work

4. Multiple Instance Learning

5. Standard Learning vs. Multiple Instance Learning Standard supervised learning Optimize some model (or learn a target concept) given training samples and corresponding labels MIL Learn a target concept given multiplesets of samples and corresponding labels for the sets. Interpretation: Learning with uncertain labels / noisy teacher

6. Multiple Instance Learning (MIL) Given: Set of I bags Labeled + or - The ith bag is a set of Ji samples in some feature space Interpretation of labels Goal: learn concept What characteristic is common to the positive bags that is not observed in the negative bags

7. Multiple Instance Learning Traditional Classification Multiple Instance Learning {x1, x2, x3, x4} label = 1 {x1, x2, x3, x4} label = 1 {x1, x2, x3, x4} label = 0 x1 label = 1 x2label = 1 x3label = 0 x4label = 0 x5label = 1

8. MIL Application: Example GPR EHD: Feature Vector Collaboration: Frigui, Collins, Torrione Construction of bags Collect 15 EHD feature vectors from the 15 depth bins Mine images = + bags FA images = - bags

9. Standard vs. MI Learning: GPR Example Standard Learning Each training sample (feature vector) must have a label Arduous task many feature vectors per image and multiple images difficult to label given GPR echoes, ground truthing errors, etc … label of each vector may not be known EHD: Feature Vector

12. Random Set Brief Random Set

13. How can we use Random Sets for MIL? It is NOT the case that EACH element is NOT the target concept Random set for MIL: Bags are sets (multi-sets) Idea of finding commonality of positive bags inherent in random set formulation Sets have an empty intersection or non-empty intersection relationship Find commonality using intersection operator Random sets governing functional is based on intersection operator Capacity functional : T A.K.A. : Noisy-OR gate (Pearl 1988)

14. Random Set Functionals Capacity functionals for intersection calculation Use germ and grain model to model random set Multiple (J) Concepts Calculate probability of intersection given X and germ and grain pairs: Grains are governed by random radii with assumed cumulative: Random Set model parameters Germ Grain

15. RSF-MIL: Germ and Grain Model x T x T T x x x x T T x x x Positive Bags = blue Negative Bags = orange Distinct shapes = distinct bags

16. Multiple Instance Learning with Multiple Concepts

17. Multiple Concepts: Disjunction or Conjunction? Disjunction When you have multiple types of concepts When each instance can indicate the presence of a target Conjunction When you have a target type that is composed of multiple (necessary concepts) When each instance can indicate a concept, but not necessary the composite target type

18. Conjunctive RSF-MIL Previously Developed Disjunctive RSF-MIL (RSF-MIL-d) Conjunctive RSF-MIL (RSF-MIL-c) Noisy-OR combination across concepts and samples Standard noisy-OR for one concept j Noisy-AND combination across concepts

19. Synthetic Data Experiments Extreme Conjunct data set requires that a target bag exhibits two distinct concepts rather than one or none AUC (AUC when initialized near solution)

20. Application to Remote Sensing

21. Disjunctive Target Concepts Target Concept Type 1 NoisyOR Target Concept Type 2 NoisyOR OR … Target Concept Type n NoisyOR Target Concept Present? Using Large overlapping bins (GROSS Extraction) the target concept can be encapsulated within 1 instance: Therefore a disjunctive relationship exists

22. What if we want features with finer granularity Constituent Concept 1 (top of hyperbola) NoisyOR Target Concept Present? AND … Constituent Concept 2 (wings of hyperbola) NoisyOR Our features have more granularity, therefore our concepts may be constituents of a target, rather than encapsulating the target concept Fine Extraction More detail about image and more shape information, but may loose disjunctive nature between (multiple) instances

23. GPR Experiments Extensive GPR Data set ~800 targets ~ 5,000 non-targets Experimental Design Run RSF-MIL-d (disjunctive) and RSF-MIL-c (conjunctive) Compare both feature extraction methods Gross extraction: large enough to encompass target concept Fine extraction: Non-overlapping bins Hypothesis RSF-MIL will perform well when using gross extraction whereas RSF-MIL-c will perform well using Fine extraction

24. Experimental Results Highlights RSF-MIL-d using gross extraction performed best RSF-MIL-c performed better than RSF-MIL-d when using fine extraction Other influencing factors: optimization methods for RSF-MIL-d and RSF-MIL-c are not the same Gross Extraction Fine Extraction

25. Future Work Implement a general form that can learn disjunction or conjunction relationship from the data Implement a general form that can learn the number of concepts Incorporate spatial information Develop an improved optimization scheme for RSF-MIL-C

26. Backup Slides

27. MIL Example (AHI Imagery) Robust learning tool MIL tools can learn target signature with limited or incomplete ground truth Which spectral signature(s) should we use to train a target model or classifier? Spectral mixing Background signal Ground truth not exact

28. MI-RVM Addition of set observations and inference using noisy-OR to an RVM model Prior on the weight w

29. SVM review Classifier structure Optimization

30. MI-SVM Discussion RVM was altered to fit MIL problem by changing the form of the target variable’s posterior to model a noisy-OR gate. SVM can be altered to fit the MIL problem by changing how the margin is calculated Boost the margin between the bag (rather than samples) and decision surface Look for the MI separating linear discriminant There is at least one sample from each bag in the half space

31. mi-SVM Enforce MI scenario using extra constraints Mixed integer program: Must find optimal hyperplane and optimal labeling set At least one sample in each positive bag must have a label of 1. All samples in each negative bag must have a label of -1.

32. Current Applications Multiple Instance Learning MI Problem MI Applications Multiple Instance Learning: Kernel Machines MI-RVM MI-SVM Current Applications GPR imagery HSI imagery

33. HSI: Target Spectra Learning Given labeled areas of interest: learn target signature Given test areas of interest: classify set of samples

34. Overview of MI-RVM Optimization Two step optimization Estimate optimal w, given posterior of w There is no closed form solution for the parameters of the posterior, so a gradient update method is used Iterate until convergence. Then proceed to step 2. Update parameter on prior of w The distribution on the target variable has no specific parameters. Until system convergence, continue at step 1.

35. 1) Optimization of w Optimize posterior (Bayes’ Rule) of w Update weights using Newton-Raphsonmethod

36. 2) Optimization of Prior Optimization of covariance of prior Making a large number of assumptions, diagonal elements of A can be estimated

37. Random Sets: Multiple Instance Learning Random set framework for multiple instance learning Bags are sets Idea of finding commonality of positive bags inherent in random set formulation Find commonality using intersection operator Random sets governing functional is based on intersection operator

38. MI issues MIL approaches Some approaches are biased to believe only one sample in each bag caused the target concept Some approaches can only label bags It is not clear whether anything is gained over supervised approaches

39. RSF-MIL x T x T T x x x x T T x x x MIL-like Positive Bags = blue Negative Bags = orange Distinct shapes = distinct bags

40. Side Note: Bayesian Networks Noisy-OR Assumption Bayesian Network representation of Noisy-OR Polytree: singly connected DAG

41. Side Note Full Bayesian network may be intractable Occurrence of causal factors are rare (sparse co-occurrence) So assume polytree So assume result has boolean relationship with causal factors Absorb I, X and A into one node, governed by randomness of I These assumptions greatly simplify inference calculation Calculate Z based on probabilities rather than constructing a distribution using X

42. Diverse Density (DD) Probabilistic Approach Goal: Standard statistics approaches identify areas in a feature space with high density of target samples and low density of non-target samples DD: identify areas in a feature space with a high “density” of samples from EACH of the postitive bags (“diverse”), and low density of samples from negative bags. Identify attributes or characteristics similar to positive bags, dissimilar with negative bags Assume t is a target characterization Goal: Assuming the bags are conditionally independent

43. Diverse Density It is NOT the case that EACH element is NOT the target concept Calculation (Noisy-OR Model): Optimization

44. Random Set Brief Random Set

45. Random Set Functionals Capacity and avoidance functionals Given a germ and grain model Assumed random radii

46. When disjunction makes sense Target Concept Present OR Using Large overlapping bins the target concept can be encapsulated within 1 instance: Therefore a disjunctive relationship exists

48. Improve Existing Code

49. Develop joint optimization for context learning and MIL

50. Apply MIL approaches (broad scale)

51. Learn similarities between feature sets of mines

52. Aid in training existing algos: find “best” EHD features for training / testing

54. How can we use Random Sets for MIL? Random set for MIL: Bags are sets Idea of finding commonality of positive bags inherent in random set formulation Sets have an empty intersection or non-empty intersection relationship Find commonality using intersection operator Random sets governing functional is based on intersection operator Example: Bags with target {l,a,e,i,o,p,u,f} {f,b,a,e,i,z,o,u} {a,b,c,i,o,u,e,p,f} {a,f,t,e,i,u,o,d,v} Bags without target {s,r,n,m,p,l} {z,s,w,t,g,n,c} {f,p,k,r} {q,x,z,c,v} {p,l,f} intersection union Target concept = br />{a,e,i,o,u,f} {f,s,r,n,m,p,l,z,w,g,n,c,v,q,k} = {a,e,i,o,u}